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Illustrations: The cover by Dan Sha- 
piro gives time the prominence it de- 
serves in all phases of College Board 
test construction, administration, and 
score reporting, and the hour the 
importance it assumes to candidates 
throughout the world as they start the 
Scholastic Aptitude Test at 9:00 a.m. 
sharp. Other aspects of college admis- 
sion are presented in pen and ink draw- 
ings by Stanley Wyatt on pages 3, 6, 8, 
9, 13, 14, 15, 16, 17, and 22. 


NEWS OF THE COLLEGE BOARD 


Spring meeting 


April 6: A morning symposium and 
an afternoon business meeting de- 
voted to the discussion of present and 
possible future activities of the Col- 
lege Board have been scheduled for 
the April 6 spring meeting. 

Among the important items to be 
considered in the afternoon will be the 
question of whether or not the General 
Composition Test (see page 2) should 
be continued in the testing program. 
Other business to be considered in- 
clude a proposal to expand the Execu- 
tive Committee so that the areas of in- 
terest within the Board membership 
may be more adequately represented 
by the committee, presentation of a 
plan for the first administration of the 
Advanced Placement Tests in the 
spring of 1956, and the reports and 
recommendations of committees. 

The morning symposium will deal 
with problems arising from emerging 
patterns of admission which are tend- 
ing to complicate the transition from 
school to college. 


Forthcoming publications 


Director’s Report: A full account of 
College Board activities in the 1953- 
54 academic year, including extensive 
data on the number and distribution 
of candidates and test score reports, 
appear in the 53rd Report of the Direc- 
tor, which will be sent to subscribers 
in March. 

In the Report Director Frank H. 
Bowles describes and appraises such 
significant steps as the Board’s estab- 
lishment of a scholarship service (see 
page 22), a National Science Founda- 
tion study of methods to reduce the 
loss of capable science students in the 
transition from school to college, and 
the adoption during the year of more 
rigorous Board membership require- 
ments. 


College Admissions, No. 2: A special 
publication which will cover the com- 
plete proceedings of the second College 
Board Colloquium on College Admis- 
sions is scheduled for March. College 
Admissions, No. 2 reproduces the four- 
day series of classes at which the ad- 
missions officers of 90 Board member 
colleges last October discussed the ef- 
fects of social, intellectual, personal, 
and financial factors on college attend- 
ance. The cost of the book is $3.50. 


Standing committees 


Vacancies filled: Appointments re- 
cently announced by Provost Samuel 
T. Arnold of Brown University, Chair- 
man of the College Board, have com- 
pleted the membership of Board stand- 
ing committees for the current year. 

The places of retiring members of 
the Committee on Research and De- 
velopment were taken by Professor 
Irving Lorge, Teachers College, Co- 
lumbia University, and Professor 
Quinn McNemar, Stanford University. 
Retiring committee members are Pro- 
fessor Andrew H. MacPhail, Brown 
University, and Professor James A. 
McClintock, Drew University. 

New members of the Committee on 
Nominations are Dr. Finla G. Craw- 
ford, vice chancellor, Syracuse Uni- 
versity; Dr. William H. Cornog, presi- 
dent, Central High School, Philadel- 
phia, Pennsylvania; Mrs. Ruth W. 
Crawford, director of admission, Smith 
College; and Hubert S. Shaw, director 
of admissions, Bowdoin College. They 
succeed Dr. Allan V. Heely, headmas- 
ter, Lawrenceville School, Lawrence- 
ville, New Jersey; Dr. Wilson Parkhill, 
headmaster, Collegiate School, New 
York, New York; Professor B. Alden 
Thresher, director of admissions, Mas- 
sachusetts Institute of Technology; 
and Annie C. Whiteside, registrar, 
Randolph-Macon Woman’s College. 


The largest number of new member 
appointments was in the Committee 
on Examinations, which now includes 
Professor Frederick B. Agard, Cornell 
University; Edwin C. Douglas, assist- 
ant to the headmaster, Taft School, 
Watertown, Connecticut; Harold 
Howe, II, principal, Walnut Hills High 
School, Cincinnati, Ohio; Ruth Jen- 
kins, headmistress, Annie Wright 
Seminary, Tacoma, Washington; Pro- 
fessor Arthur E. Jensen, Dartmouth 
College; Professor Thomas C. Men- 
denhall, Yale University; and Pro- 
fessor Denham Sutcliffe, Kenyon Col- 
lege. Retiring members of the com- 
mittee are Dr. Robert G. Andree, 
headmaster, Brookline (Massachu- 
setts) High School; Professor Harry 
D. Berg, Michigan State College; Dr. 
William H. Cornog, president, Central 
High School, Philadelphia, Pennsyl- 
vania; C. William Edwards, director 
of admission, Princeton University; 
Dean Everett L. Hunt, professor of 
English, Swarthmore College; Allegra 
Maynard, headmistress, Madeira 
School, Greenway, Virginia; and Don- 
ald D. Walsh, director of senior studies 
and Spanish department head, Choate 
School, Wallingford, Connecticut. 


Visiting representative 


To see urban colleges: T. Leslie Mac- 
Mitchell, former assistant dean of ad- 
missions at New York University, has 
joined the staff of the College Board 
on a temporary basis to visit member 
and non-member urban colleges dur- 
ing the spring months. The purpose of 
his visits will be to collect information 
on the admissions procedures and 
problems of degree-granting afternoon 
and evening study programs. 

Mr. MacMitchell recently resigned 
from the Brooklyn Dodgers organiza- 
tion in which he served for one year 
as executive assistant to the president. 








Should the General Composition Test be continued ? 


The General Composition Test, grown 
familiar during the past four years as 
an idea, then as an experiment, is scru- 
tinized for the first time in the two 
articles which follow in terms of its 
performance as a regular College 
Board test. 

There are two articles because at 
least two positions may be taken on the 
test and the results of its first adminis- 
tration. The positions are far apart, 
based on differences in theory which 
could have been advanced just as 
firmly five years ago before there was 
a GCT, and before the quest for “a test 
of writing ability” could be expressed 
as a debate on specific testing proce- 
dures or statistical interpretations. 

In simplest and most extreme lan- 
guage, it may be said that those who 
espouse one or the other of these two 
viewpoints do not see the same thing 
when they look at the cct. They not 
only see the other side of the same 
coin; the other side is stamped with a 
different value in another language. 

Presented with any test which is to 
be introduced as a college entrance 
examination, the spokesman for one 
point of view will claim that the test 
should measure the kinds of things 
that are taught in school, and should 
produce the best possible prediction of 
the candidate’s success in college. This 
can be done best, he will say, by tests 
that are “reliable,” that is, may be 
scored so consistently that the same 
score always represents very close to 
the same degree of achievement, and 
that they are “valid,” that is, forecast 
how well or poorly the student will do 
academically in college. 

The test is, first and foremost, a test, 
and the ways in which its reliability 
and validity characteristics affect its 
use for college admissions purposes 
are considered of essential importance. 
Whether it is an objective test, or an 
essay test like the GcT, it must be effec- 


tive as a test which measures consist- 
ently and accurately. 

The opposite opinion denies that 
any weakness the essay test may have 
as an instrument of selection should 
override, its superior value as an aid to 
instruction. Proponents of this persua- 
sion point to the interest and enthusi- 
asm of schools which have used the 
cct and the unexpectedly large num- 
ber of candidates who took the test the 
first time it was scheduled by the Col- 
lege Board. If teachers armed with a 
good essay test are thereby enabled to 
improve the general state of English 
composition and provide the colleges 
with a more literate candidate, it is 
argued, the application of validity and 
reliability standards to select students 
on the grounds of precise differences 
in writing ability becomes progres- 
sively less important. 

Moreover, it is unfair and unfeel- 
ing, in a most vital sense, to measure 
an essay test according to objective 
test standards and only by them. The 
best test of writing, it is proposed, is 
writing itself; the best judges of Eng- 
lish composition are those who teach 
it and who are selected by the College 
Board to work together on this test. 
They do not expect to see a perfect 
essay test any sooner than a purple 
cow, but they do believe the ccr is a 


good test which will encourage im- 
provements in English teaching. 

The question which the College 
Board will face at its meeting this April 
is whether the unique educational vir- 
tues claimed for the GcT compensate 
for an apparent loss in reliability 
which is conspicuous in contrast to 
other College Board tests. If the Board 
accepts the GcT as a permanent fea- 
ture of its admissions test battery, it 
must also accept the risks inherent in 
a test which does not behave as con- 
sistently as the other tests in the pro- 
gram. 

The recommendation of the officers 
and staff of the Board and the Educa- 
tional Testing Service is that the GcT 
be offered at least one more year after 
this May, when the two-year trial pe- 
riod will expire, and that certain stud- 
ies be conducted during that time to 
complete general understanding of the 
test. These would probably include an 
attempt to appraise the educational 
value of the test, a comparison of the 
performance of the cct with that of 
the Board’s objective English Compo- 
sition Test, the exploration of possible 
ways to improve GcT reliability, and an 
attempt to adjust scores for recogniz- 
able differences in the leniency or 
strictness. with which certain readers 
habitually grade essays. 


The test fails as an entrance examination 


BY RICHARD PEARSON 


The General Composition Test had its 
genesis in a report of a special College 
Board Committee on English Testing 
which met in 1949-50 and expressed 
its concern that “when the essay type 
test was abandoned [in the Board’s 


program | there ceased to be any test- 
ing of certain crucially important edu- 
cational values.” The committee 
“dreaded that unless these values could 
be re-emphasized, or possibly even re- 
discovered, the effect on American 


education would be most unfortunate.” 

The result of this concern was a two- 
hour test of writing ability which re- 
quires the student to read background 
materials on the topic presented, out- 
line an essay on the topic, write the es- 
say, and then summarize its theme 
briefly. This is the test which was of- 
fered last May for the first time in the 
regular College Board program and 
which is scheduled for a second, and 
possibly final, administration in May, 
1955. The future of the ccr will de- 
pend upon action to be taken by the 
Board at its spring meeting on April 6, 
when it will become necessary to re- 
consider the two-year trial period ap- 
proved by the Board in 1953. 


From tryout to regular test 


The events which led up to the in- 
troduction of the test included experi- 
mental administrations of tryout forms 
to 800 students from 29 public and in- 
dependent schools in the spring of 
1951 and to 800 freshmen at four col- 
leges in the fall of the following year. 
The experiments developed procedures 
for the selection and presentation of 
essay topics and for the reading of 
students’ papers so that their scores 
would be reported in terms which 
would be meaningful to admissions of- 
ficers. During this time, in response to 
inquiries about the test, copies of the 
tryout forms were also made available 
at cost to interested schools for their 
administration, scoring, and use. In 
the spring of 1952 more than 300 
schools ordered more than 8,000 cop- 
ies of the test. The following spring, 
when a second form of the test was 
offered, about 700 schools ordered 
more than 14,000 copies. 

Among the factors to be considered 
by the Board in April and by the ap- 
propriate Board committees which 
early in March will consider whether 
or not to recommend continuation of 
the test is a sizable fund of information 
gleaned from the first regular adminis- 
tration and studied by the Educational 
Testing Service. The comments which 
follow summarize those studies and 
represent the ETS interpretation of 
data made available by that one ad- 
ministration. 

A total of 4,272 candidates took the 
ccT in May 1954 of whom 96% were 
preliminary or junior year candidates, 





86% were students from independent 
schools, and 64% were girls. A special 
study of the candidates tested by the 
ETS office in Princeton indicates that 
30 colleges which received GcT score 
reports for 20 or more candidates ac- 
counted for a total of 2,610 reports. 
Of these, 1,630 score reports, or 62%, 
were sent to the women’s colleges com- 
prising the Seven College Conference 
group, although apparently not as a 
requirement for admission. Except for 
the Seven Colleges and possibly Con- 
necticut College, which received 121 
score reports, no college appears to 
have requested scores on this test gen- 
erally for junior year applicants. A 
total of 80 secondary schools with 20 
or more candidates, of which 74 
schools were independent and six were 
public, requested and received their 
GcT scores. These schools were repre- 
sented by 3,180 reports, or 74% of the 
students taking the test. 

At the same time that the cct was 
offered in the regular May series, al- 
ternate forms of the test were made 
available to the colleges at nominal 
cost through the Board’s regular place- 
ment testing program. During 1953-54 
two colleges ordered the cct for place- 
ment use and a total of 143 tests were 
administered. In the current academic 
year one college has ordered 44 cop- 
ies. It should be noted that the offer- 
ing of this test in the placement pro- 
gram did not include reading of the 
essays by the College Board. 

Reading of the May 1954 examina- 
tion employed careful techniques de- 
veloped during the two experimental 
reading conferences to secure a com- 





*Barnard, Bryn Mawr, Mount Holyoke, 
Radcliffe, Smith, Vassar, and Wellesley 
Colleges 


mon understanding of grading stand- 
ards. An intensive period of discus- 
sion, training, and practice reading 
was held at the beginning and the 
training process was continued 
throughout the conference—training 
not to impose fixed standards of excel- 
lence, but to secure full cooperation 
from each reader in the definition of 
these standards. Each essay was read 
twice by two different readers and day- 
to-day records of the progress of the 
reading were maintained and evalu- 
ated. The reading conference, held at 
Haverford College in the last week of 
June, was described as follows by 
Earle G. Eley, the chief reader and 
guiding spirit of the test since its be- 
ginning: 

“One question was whether 4,300 
essays could be read in the allotted 
time. This question was answered in 
the affirmative. The average time for 
two readings of an essay plus a third 
reading when necessary during the 
two earlier experiments had been 2714 
minutes. Thus it was estimated about 
30 minutes per essay for the Haver- 
ford Conference. However, neither of 
the two previous experiments had al- 
lowed for the actual reading after com- 
pletion of training to run for more 
than three days. There was some indi- 
cation that the speed of reading might 
increase significantly if a conference 
were extended to allow for as many as 
five days of uninterrupted reading. 
This actually proved to be the case. It 
was estimated that during the time 
available at Haverford, as many as 
6,000 essays could have been read. 

“Another question—could we de- 
velop a spirit of unity among the read- 
ers—it was also possible to answer in 
the affirmative. Although we were not 
able to hold the pre-dinner meetings 








that had during the two experimental 
conferences been so large a factor in 
developing rapport, we were pleased to 
find that even so large a group as 52 
readers by the second or third day had 
taken on a quality of enthusiasm which 
made for a spirit of unity in the con- 
ference. The readers pronounced a ver- 
dict in favor of the test which was al- 
most unanimous. Despite the grueling 
task of reading essays for six hours a 
day, day after day, readers continued 
to be interested in the problems of 
judgment and continued to discuss 
among themselves problems of reading 
during the morning and afternoon cof- 
fee breaks. 

“In the earlier conferences dealing 
as we did with no more than 800 cases, 
two clerical workers and three admin- 
istrative workers using a system of 
hand entries both for computing final 
grades and for computing our contin- 
uous statistical check on reader agree- 
ment proved feasible enough. At the 
Haverford Conference, it became clear 
that when the number of candidates 
goes into the thousands a system of 
hand entries is not feasible even 
though we did manage at this confer- 
ence to keep up after a fashion.” 

The possibility of undertaking this 
clerical work by means of 1BM tabu- 
lating equipment has subsequently 
been studied by Ets and it seems likely 
that in the next reading conference 
the use of this equipment can be 
counted on to provide the following 
clerical services: (1) the automatic 
resolution of grades assigned by the 
first and second readers where differ- 
ences between the two readings fall 
within pre-determined limits; (2) the 
identification of papers where dis- 
agreements between the first two read- 
ers are sufficiently great to require a 
third reader; (3) summary informa- 
tion, reader by reader, on the work of 
a given day which will be available by 
the morning of the following day for 
use in training. It appears that as 
many as 6,000 essays could be hand- 
led with a combination of the same 
number of readers and an improve- 
ment in our clerical procedures. It is 
doubtful, because of the difficulty of 
obtaining readers before the close of 
school, that the scores can be re- 
ported in substantially less time than 
the two months required for last May’s 
test. It also seems that an increase to 


The test itself 


The General Composition Test is a two-hour test of writing ability, 
which will be offered only at the May series. Candidates taking this test 
may take only one other Achievement Test at this series. 

The essay topic for this test is a general problem of wide interest. The 
student is given reading materials for background on the topic pre- 
sented. After reading the background materials, the student is asked to 
prepare an outline for his essay, to write the essay, and to summarize 
its theme briefly. 

Two problems which have been used as topics are: “Should women 
be given the same educational and professional opportunities as men?” 
and “Is there a conflict between science and human values?” 

The preliminary instructions to the student follow: 

Directions: Today you will be given a problem stated in general terms 
and a set of reading materials dealing with one case in which this prob- 
lem has come up. After you have read the materials carefully, you will 
be asked to write an essay stating as well as you can what you think 
about the problem. 

Your essay will be judged impartially by qualified readers according 
to five different qualities. These five qualities are: 


. Mechanics. Your ability to use correct forms of grammar, to punctuate 
and spell correctly. 

. Style. Your ability to state your ideas clearly and effectively in language 
which is appropriate to your essay. 

. Organization. Your ability to arrange the parts of your essay in such a 
way as to make your ideas and purpose clear to your readers. 

. Reasoning. Your ability to support your conclusions with evidence and 
to reason in such a way as to convince your readers of the soundness of 
your conclusions. 

. Content. Your ability to supplement the reading materials given with 
material drawn from your general reading, your school studies, and 
your own life. 


You have two hours for the complete examination. Be sure to read 
all the material and all parts of the writing assignment before you begin 
to write. It is suggested that you spend about twenty minutes reading 
the materials. In the hour and forty minutes remaining you should plan 


and write your essay. 


College Board Tests, 1954-55, The College Entrance Examination Board Bul- 


letin of Information, p. 35. 


10,000 or 15,000 candidates could be 
scored by a proportionate increase in 
the number of readers and adaptation 
of the tabulating equipment. Whether 
a greater number could be handled 
with these procedures is uncertain. 


The question of reliability 


An important measurement charac- 
teristic of the GcT is the extent to which 
the readers agree on the assignment of 
scores to a given paper, that is whether 
they agree on what constitutes a high- 
scoring or a low-scoring paper. The 





question is not whether the readers 
agree, for it may be assumed that there 
will be some disagreement, but rather 
how much they disagree and if this dis- 
agreement falls within tolerable limits. 

The ccrT score scale reported to 
schools and colleges ranges from 1 
(low) to 4 (high) points. A score of 
“1” assigned by a reader indicates the 
reader’s belief that the student is un- 
prepared to do college composition. 
Similarly, a “2” indicates the reader’s 
judgment that the student needs addi- 
tional training, although he may be 
able to do college work, a “3” that he 





could do satisfactory work but would 
benefit materially from an introduc- 
tory writing course, and a “4” that he 
wrote well enough so that he might be 
excused from introductory writing 
courses. These were the “labels” used 
in the reading process; they do not 
imply any inherent recommendations 
for selection of placement. The scores 
are reported in the categories of 
Mechanics, Style, Organization, Rea- 
soning, and Content, each paper re- 
ceiving a score in each category. Dur- 
ing the early experimentation with the 
ccT, these five category scores were 
added for each student and a com- 
posite GCT score was derived. Since 
the Board’s Committee on Examina- 
tions decided not to report composite 
scores for the May 1954 administra- 
tion, we have not obtained statistics 
for the composite in all of the studies 
which follow. 

In an attempt to determine the ex- 
tent to which the readers agreed in 
scoring, Frances Swineford of ETS 
traced a random sample of 370 papers 
through the reading process.’ Selected 
to be typical of all papers read last 
summer and not differentiated from 
the others at any point during the 
reading, these essays received the regu- 
lar two independent readings. This 
step is carefully controlled by an elab- 
orate routing system to insure that 
each reader reads “against” every 
other reader approximately to the 
same extent throughout the confer- 
ence. It should be observed that the 
readers use a more refined score scale 
at this point than is used to report final 
scores. Permissi.n is given to use 
“borderline” categories when, in the 
reader’s judgment, a paper deserves a 
“low 4” or a “high 2.” The effect of 
this is to expand the original 4-point 
scale to a 10-point scale which is sub- 
sequently contracted back to a 4-point 
scale when final scores are assigned. 

It will be helpful now to refer to the 
370 papers in Dr. Swineford’s study 
after the scores of the first two inde- 
pendent readings were obtained, using 
the 10-point scale. To simplify the 
presentation, we have selected only two 
of the five categories, Style and Organ- 
ization. The Style category was the 


*“Reading Reliability of the General Com- 
position Test, Form H,” Statistical Report 
No. 54-34, October 1954, Educational Test- 
ing Service, Princeton, N. J. 


one in which greatest agreement 
among the readers was secured; the 
Organization category was among the 
categories most difficult to read. Dr. 
Swineford’s comparison of the first 
and second readings of these catego- 
ries for the 370 papers in the sample 
yielded the following information: 


Number of papers Style Organization 
With complete agree- 


ment between two 


EN dnd kaaeninn 3 92 78 
With disagreement of: 
QUO RONE Ficceicicesi 100 90 
TWO ROHS 6 60 os cee 91 81 
Three points ......... 59 65 
Four points .......... 22 35 
Five points ‘ dita Me 10 
SOND. >. - ededcwes 2 10 
Seven points ......... — 1 


Eight points ......... — — 
PN necccavies — — 


Total number of papers 370 370 


The product-moment correlations 
summarizing the amount of agreement 
between first and second readings are 
.57 for Style and .43 for Organization. 
Corresponding values for the other 
categories are .53 for Mechanics, .42 
for Reasoning, and .44 for Content. 


Reconciling disagreements 


The second step in the reading proc- 
ess was a reconciliation of some of 
these disagreements between first and 
second readers. This reconciliation 
was made solely with reference to dif- 
ferences in first and second reading 
scores; no attempt was made to re- 
read any papers in this step. The pro- 
cedure is as follows: 

All one-point disagreements are re- 
solved by (1) accepting the integral 
score as the final score, in cases where 
one reader assigns an integral score 
and the other reader assigns an adja- 
cent borderline score; and (2) deter- 
mining the final integral score by an 
inspection of final scores assigned to 
other categories when both readers 
assign scores on the same borderline 
but indicating adjacent integral scores. 
An example of the first of these is a 
first reading score of “3” and a second 
reading score of “low 3.” The final 
score would be the integral score “3.” 
An example of the second of these is a 
first reading Style score of “low 3” and 
a second reading Style score of “high 
2.” The integral score of “2” or “3” 


would be assigned the paper as a final 
Style score, depending upon the pre- 
ponderance of “2’s” and “3’s” among 
Mechanics, Organization, Reasoning, 
and Content for that paper. 

All two-point disagreements are re- 
solved by (1) accepting the integral 
score as the final score, in cases where 
one reader assigns an integral score 
and the other reader assigns a border- 
line score which is adjacent to the next 
higher or lower integral score; and 
(2) accepting as the final score that 
integral which falls between two bor- 
derline scores, in cases where one 
reader uses the upper border and the 
other reader uses the lower border of 
the same integral score. An example 
of the first of these is a “2” and a “low 
3”; a final score of “2” would be as- 
signed this paper, without further 
reading, despite the fact that the sec- 
ond reader indicated a preference for 
a “3.” An example of the second of 
these is a “high 3” and a “low 3”; the 
final score in this case is “3.” 

Some three-point disagreements are 
resolved when two borderline ratings 
were assigned by the two readers and 
when one of the borderlines indicated 
a preference for a given integral score 
and the other indicated a preference 
for the next higher or lower integral 
score. That is, a “high 2” and a “high 
1” would be assigned a final score of 
“2,” despite the fact that the second 
reader indicated a preference for a 
aed ng 

The effect on the 370-case sample of 
this reconciliation process is summar- 


ized below: 


Number of papers Style Organization 
With complete agree- 

ment between two 

WOMEN. fh bs ck 508 92 78 
With disagreements 

which have been 

resolved by adjusting 

scores of two 

WORE said ness o cee 208 179 
With remaining dis- 

agreements of: 


Three points ......... 42 57 
Four points .......... a ee 
FIO MONEE. «cece eee 4 10 
SU NOE 8 Sid's Sones 2 10 
Seven points ......... = 1 
Eight points ......... — — 
Nine points .......... — — 


Total number of papers 370 370 


The final step in the reading process 








is to refer the papers with disagree- 
ments remaining between first and sec- 
ond readers to a third reader. Nor- 
mally a third reading is sufficient to 
resolve these discrepancies and it was 
for the remaining 70 Style and 113 Or- 
ganization disagreements in the sam- 
ple. 

In other words, in their reading of 
370 essays and scoring them on a 10- 
point scale, the first and second read- 
ers were in complete agreement on the 
Style score in 25% of the cases and on 
the Organization score in 21% of the 
papers. If the definition of “agree- 
ment” is broadened to accommodate 
the reconciliation of scores showing 
one or two, and in a few cases, three 
points of difference, the agreement for 
Style rises to 81% and for Organiza- 
tion to 69%. The agreement reached 
100% in both cases with resolution by 
a third reading. 


Strict or lenient? 


Which two readers graded a paper 
was also found to have an important 
bearing on the reported scores. This 
factor was recognized in planning for 
the conference because it seemed likely 
that a paper scored by the strictest and 
the most lenient of readers might have 
to be referred to a third reader, while 
scorings by two readers who were both 
strict, both lenient, or both moderate 
would by reason of that pairing, rather 
than the quality of the student’s writ- 
ing, be in close or complete agreement. 
Great care was taken in the selection 
of experienced readers and in the 
training part of the conference to 
avoid variations in scoring caused by 
the readers and Dr. Swineford sought 
to discover how successful these pre- 
cautions had been. Twenty-four papers 
were selected at random, none from 
the sample of 370 just described, and 
each was read by 15 readers according 
to a most detailed routing design 
which protected the integrity of the 
study. Since each paper was read by 
15 readers, there were 105 instances 
for each paper where different pairs 
of readers were involved. Thus, it is 
possible to know how the papers would 
have been scored by one pair of read- 
ers, then by another pair, and so on 
through 105 pairs. 

Taking the first of two extreme pa- 
pers for illustrative purposes—Paper 





A paper scored by a strict and a lenient 
reader might need a third reading 


A, which was easy to score—we see be- 
low what actions would have been 
taken in the 105 cases, at the step in the 
reading process where scores of first 
and second readers are compared. The 
scores are given in the five categories, 
M (Mechanics), S (Style), O (Organ- 
ization) , R (Reasoning), and C (Con- 
tent). 


Final grade n8) 8-8 Se 
Dinth ind taceeneseken 66 101 65 72 78 
Bip ascndintbanie tsaxninn 24 4 10 17 #5 
Diiad wetenedners dake —- -—- -—- -—- — 
Di cites bc pick —_—- —- —- — 
Third reader ........ 1, — 30 16 22 
Reader pairs ........ 105 105 105 105 105 


Paper A would have received a 
grade of “4” in Mechanics 66 times 
out of 105, depending solely on which 
two readers were given the paper dur- 
ing the first and second readings. 
Twenty-four times out of 105, a final 
Mechanics grade of “3” would have 
been assigned to the papers as a result 
of the first two readings. The total of 
these, or 90 times out of 105, gives the 
number of times when agreements be- 
tween first and second readings were 
within previously defined limits and 
resulted in no third reading. 

The following actions would have 
been taken on Paper B, which was dif- 
ficult to score: 


Final grade MS °OD -2&.C 
OF ise cen cs wad 2—--—--—--— 
Pek Neiuk beak oxy cake 28 15 24 36 43 
Bits ditebackecowens 37 30 49 12 23 
Licpewntéayeraeasicws 2 38. 22 @ 
Third reader ........ 37 54 30 51 37 
Reader pairs ........ 105 105 105 105 105 


The variations in final scores which 
would result if Paper B were given 
first and second readings by different 
pairs of readers are considerable. A 
final Mechanics score over the entire 
scale is possible, depending solely 
upon which two readers were given 
the paper. Final scores in Style, Organ- 


ization, Reasoning, and Content over 
three of the four scale points are pos- 
sible. And even though Paper B is diffi- 
cult to score, there is no firm probabil- 
ity that a third reading will be given 
this paper under existing reading pro- 
cedures. It must be concluded from 
this evidence, which is supported by 
more general statistics from Dr. Swine- 
ford’s study, that which two readers 
participate in the first and second read- 
ings has a decided influence on the re- 
ported scores. 

The crucial question in this consid- 
eration of reader agreement is what 
would happen if all papers from May 
1954 were graded a second time by 
means of a second reading conference. 
To what extent would the same papers 
receive the same scores? A second 
conference was not held but Dr. Swine- 
ford did estimate what the results 
would have been from data collected 
in the Haverford reading. Her general 
conclusion was that “the reading reli- 
abilities for the reported category 
scores probably range from the high 
.50’s to about .70, when the scores are 
based on ratings by two or three read- 
ers.” Mechanics and Style generally 
showed a greater reader agreement 
than did Organization, Reasoning and 
Content. Although a composite GcT 
score was not obtained at Haverford, it 
is not likely that the reading reliability 
of this score would exceed .75. 


The difference in practice 


The implications of this finding may 
be seen by posing an operational ques- 
tion facing a school or college official: 
“If I am considering two students 
each of whom took the cct last May, 
how much of a difference in score be- 
tween the two should there be before I 
can say with confidence that the differ- 
ence did not arise because of reader 
disagreement?” Remembering that 
GCT scores are reported on a four-point 
scale, the evidence from Dr. Swine- 
ford’s study suggests that the state- 
ment could not be made with confi- 
dence unless the score difference were 
two points—one less than the maxi- 
mum possible for any two GCT scores. 
It appears evident that reader disagree- 
ment was a major source of error in 
the GcT scores reported to schools and 
colleges and that those of individual 
students should be used with that in 


mind. It is also likely that it would 
take 10 to 15 readings of each paper 
to eliminate that source of error, a sit- 
uation which suggests that while some 
improvement in reading reliability 
may be achieved by future refinements 
in the reading process, there is little 
hope that these can altogether solve a 
serious problem. 


The question of validity 


A second important measurement 
characteristic of the GcT is the extent 
to which the scores reported for indi- 
vidual students agree with independ- 
ent estimates of the students’ writing 
ability. In this case, the question is 
whether the scores “make sense” — 
whether they rank the students in a 
way which would be meaningful to 
teachers who are presumed to know 
which of their students are better than 
others in their writing ability. 

The application of this type of an 
analysis to any test is both comprehen- 
sive and ambiguous. It is comprehen- 
sive because it considers both the er- 
rors arising from reader disagree- 
ment and those which result if the 
students are not challenged equally to 
turn in their best performance. It is 
ambiguous because one fallible meas- 
ure of writing ability (the GcT) is set 
against another (teacher estimates). 
Given less than unanimity, the ques- 
tion of where the difference lies inevi- 
tably occurs. Such an analysis of the 
GCT scores was made, however, in an 
attempt to discover if the GcT is really 
a valid measure of writing ability.’ 

To obtain an independent appraisal 
of writing ability, Dr. Marjorie Olsen 
of Ets asked teachers at 11 secondary 
schools to rate their students on the 
same five categories used in scoring 
the cctT, basing their ratings on obser- 
vations extending over the school year, 
not on any single factor, and without 
prior knowledge of the ccT scores. The 
ratings at nine schools were supplied 
by the English teacher, and at the other 
two schools by the English teacher, 
with one or more teachers of other sub- 
jects collaborating. At the same time, 
junior year English and average 
grades for these students were ob- 





*“The Validity of the College Board Gen- 
eral Composition Test,” Statistical Report 
No. 55-4, January 1955, Educational Test- 
ing Service, Princeton, N. J. 


tained from school records as supple- 
mentary criteria. A total of 617 stu- 
dents were involved in the study. 

The evidence from Dr. Olsen’s study 
on how well the cct scores predicted 
the teachers’ estimates is contained in 
the correlations between Mechanics 
score and Mechanics rating, Style 
score and Style rating, etc. The aver- 
ages of these correlations over the 11 
schools are as follows: 


Ratings: Pe. Soh) Oe ee 
GCT 

(Corresponding 

GOUEF ec vcess 43 41 29 «36 = 33 


By way of contrast, it may be shown 
how well the teacher estimates might 
have been predicted if, instead of se- 
lecting the “appropriate” GCT score 
(i.e., the same category the teacher was 
rating), the ccT score showing the 
highest actual relationship were used. 
The average correlations are as fol- 


lows: 
Ratings: ye 2 © & < 
GCT 
(Most predictive 
| ee 43 41 34* 37* 37* 


The entries with asterisks indicate 
cases where “inappropriate” GCTscores 
showed higher relationships with the 
criterion than did “appropriate” CCT 
scores. The Organization rating was 
better predicted by three other GcT 
scores (Mechanics, Style, and Reason- 
ing) than it was by the Organization 
score. The Reasoning rating was 
slightly better predicted by the Style 
score than it was by the Reasoning 
score. The Content rating was better 
predicted by the Style and Reasoning 
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Richard Pearson 


director of the College Board test program, 
occupying a post in which he represents 
each organization to the other and the best 
interests of both. Mr. Pearson expresses the 
well-considered opinion of the Ets in report- 
ing its analysis of the General Composition 
Test. 


scores than it was by the Content score. 
The Style score, in fact, was the best 
predictor of the five in the case of 
every rating but Mechanics, where it 
was second best. The Style and Me- 
chanics scores, it will be remembered, 
also showed higher reading reliabili- 
ties than the other three scores. 

Admittedly, some of these differ- 
ences are small and may possibly have 
arisen by chance. Nevertheless, Me- 
chanics and Style are generally con- 
sidered to be quite tangible whereas 
Organization, Reasoning, and Content 
are more elusive. Those who have stud- 
ied these results at ETS consider it sig- 
nificant that Mechanics and Style held 
up well on this line of inquiry, whereas 
Organization, Reading, and Content 
showed evidence of some confusion. 

The Scholastic Aptitude Test Verbal 
scores were also obtained for the stu- 
dents in this study,’ with the thought 
that they might be helpful if ambigui- 
ties appeared between the GcT scores 
and the teachers’ ratings. When this 
proved to be the case, the sat scores 
were related to the teachers’ ratings, 
and the following average correlations 
were obtained for the “appropriate” 
GCT scores: 


Ratings: BG: @ 2 Ve 
GUEEAE  cxdcax eee 48 57 49 60 58 
GCT 
(Corresponding 

| ee een 43 41 29 36 = .33 


This showed that the sat, without 
exception, predicted the ratings better 
than did the appropriate ccrT score. 
Since the ratings can be predicted, it 
would seem that their use as criteria in 
the present analysis is justified. 

Correlational evidence not pre- 
sented in detail here indicates that each 
GCT score except Mechanics is more 
closely related to sat-v than it is to 
the corresponding teachers’ ratings. In 
the case of Mechanics, however, the 
GCT score shows a closer relationship 
with the teachers’ ratings. 

We have previously considered only 
the category scores of the cct which 
were reported for last May’s test, but 
these can be summed in whole or in 
part in order to yield composite scores. 
In her analysis, Dr. Olsen includes a 


*We would have liked to include the 
Board’s Achievement Test in English Com- 
position in this analysis but there was not a 
sufficient number of candidates also taking 
this test last May. 








sum of Mechanics and Style and a sum 
of all five category scores, reporting 
their correlations with a composite of 
the teachers’ ratings as follows: 


Composite 

Ratings 
MMS ne aes 48 
M+S+0+R-4C....... 51 


It is apparent from these figures 
that the Mechanics and Style catego- 
ries contribute most of the GcT predic- 
tion of teachers’ ratings. The correla- 
tion for the composite which includes 
only these categories is .48. When Or- 
ganization, Reasoning, and Content 
are added in a total composite GCT 
score, the correlation increases only 
to .51. 

In addition to the teachers’ estimates 
of writing ability, the study obtained 
the students’ junior year English 
grades and their average grades over 
all courses for that year. The averages 
over the 11 schools for correlations 
between GcT and SAT scores and these 
grades were: 

uy. S, 8 2.6. BABY 
English 
grade... 43 .48 .36 43 39 .67 


Average 
grade... 37 42 31 35 33 5B 


It may be noted that the saT is the su- 
perior predictor for these two criteria 
and that both cct and saT show higher 
relationships with English Grade than 
with Average Grade. 

It was also of interest to discover 
what improvements in prediction 
would occur if a combination of the 
cct and sat-v were used. Dr. Olsen re- 
ports the following correlations: 


Composite 
Ratings 
GCT composite only ......... 51 
UD 6 os sak ale tubers 65 
Simple combination of both ... .66 
Best-weighted combination of 
Deakins aC cthiens saebe 68 


This means that if a teacher or an 
admissions officer assumes each of the 
tests should have equal weight in the 
combination, he obtains a correlation 
of .66, an increase of .01 over the cor- 
relation using SAT-V alone. If he uses 
a “best-weighted” combination, de- 
rived uniquely for his school or col- 
lege using standard statistical tech- 
niques, such as was done for each of 
the 11 schools before averaging them, 
he may get a correlation of .68, an in- 


crease of .03 over the correlation using 
SAT-V alone. It is evident that the GcT 
adds very little to the sat-v. 

The most significant finding from 
both of the foregoing studies is the ob- 
vious weakness of the GCT score cate- 
gories of Organization, Reasoning, 
and Content. This weakness was dem- 
onstrated by the relative lack of cor- 
respondence between these scores and 
the appropriate teachers’ ratings. The 
fact is that readers on the one hand, 
and teachers on the other do not seem 
to agree on what constitutes Organi- 
zation, Reasoning, and Content when 
they are asked to define these in terms 
of student performance. This weakness 
was also apparent when we inquired as 
to the relative prediction obtained 
from a composite GcT score derived 
from Mechanics and Style as compared 
with a composite GcT score derived 
from Mechanics, Style, Organization, 
Reasoning, and Content. When we pur- 
sued this line of inquiry we found that 
prediction from a combination of Me- 
chanics and Style was almost as good 
as prediction from a combination of 
all five category scores. Further, this 
weakness is consistent with the results 
of the reading reliability work where 
it was found that the readers showed 
a greater tendency to disagree on the 
Organization, Reasoning, and Content 
categories than they did on the Me- 
chanics and Style categories. The 
weight of this evidence indicates un- 
mistakably that the ccT is largely a 
measure of Mechanics and Style and 
cannot presume to be an effective mea- 
sure of Organization, Reasoning, and 
Content. 

The question which may be asked 
in consideration of the data presented 
above is whether the Gct, alone or in 





If he uses a weighted combination . . 


combination with the saT-V, is a use- 
ful predictor for college guidance or 
admissions. Let us assume that a col- 
lege teacher wished to place the 617 
students from this study into four 
classes: (1) the best writers; (2) the 
next-best writers; (3) the next-to-the- 
poorest writers; and (4) the poorest 
writers. Assume further that he would 
like to use, as his criterion of writing 
ability, the school teachers’ ratings 
from this study, because they are based 
on day-to-day observations of the stu- 
dents’ work. He does not have these 
ratings but can estimate them from 
either the GCT total scores, or the SAT-v 
scores, or a simple combination of the 
two. How many students would be 
slightly or seriously misclassified 
under each alternative ? 


Using the scores 


If the college man used the ccr total 
alone, he would be “right” in 269 of 
the 617 cases; these students would 
enter a class indicated by their 
school teachers’ ratings. He would be 
“somewhat wrong” in 298 cases; these 
students would be misplaced by one 
class from that indicated by the rat- 
ings. He would be “badly wrong” in 
50 cases; these students would be mis- 
placed by two or three classes, from 
that indicated by the ratings. 

If our college teacher used the sat-v 
alone, he would be “right” in 297 
cases, “somewhat wrong” in 289 cases, 
and “badly wrong” in 31 cases. For 
the combination of GcT and sAT-v, he 
would be “right” in 307 cases, “some- 
what wrong” in 272 cases, and “badly 
wrong” in 38 cases. 

Two conclusions can be drawn from 
this enumeration of students from Dr. 
Olsen’s study. The first is that use of 
the sat-v, rather than the GcT, would 
result in more students properly placed 
and fewer students improperly placed 
in classes sectioned according to this 
criterion of writing ability. The sec- 
ond conclusion’ is that use of a simple 


°The reader may wonder why the “best- 
weighted” combination was not used in this 
comparison. The reason is that there are 
technical grounds for suspecting that the 
procedure introduces a spuriously high de- 
gree of correlation, which would not be 
maintained if the “best-weights” were ap- 
plied to other groups of students. In the 
absence of evidence to the contrary, a more 
conservative position has been taken. 








A concern for the individual candidate 


combination of satT-v and GCcT, rather 
than sAT-v alone, would result in a 
slightly greater number of properly 
placed students but might also result 
in increasing the number of badly mis- 
placed students. 


The basic issues 


The writer is unable to close this re- 
port without commenting briefly on 
some of the issues raised by the in- 
clusion in the Board program of an 
essay test of writing ability. The first 
of these has to do with the stringency 
with which one interprets the available 
evidence dealing with measurement 
characteristics of the cct. The concern 
of Ets with reader agreement is not 
academic nor is it an attempt to harass 
a hard-working group of able readers. 
Our interpretation of this evidence is 
related directly to a concern for the 
individual candidate. From the time 
he registers for the Board tests until 
his scores are released to colleges and 
schools, we feel obliged to impose 
standards of near-perfect accuracy; 
the intermediate processes are, or 
should be, under rather complete con- 
trol. 

If we are asked to attest to the ac- 
curacy of reported scores, as we fre- 
quently are by candidates, schools, or 
colleges, we welcome the assurances of 
procedures which, when repeated, 
yield essentially the same results. 
Further, as test people, we know from 
experience that high consistency in as- 
signing scores to individual candidates 
is an attainable goal when there is 
general understanding concerning the 
task the candidate is asked to per- 
form and what constitutes satisfactory 


or unsatisfactory performance. Any- 
thing less than high consistency of 
scoring implies that we have not at- 
tained this general understanding and 
hence are operating in the realm of 
chance, rather than design. 

A second issue raised by the GcT is 
its effect on instruction in the sec- 
ondary schools. The present report 
does not present evidence bearing di- 
rectly on this important issue. It is 
likely that carefully conceived research 
could contribute much helpful infor- 
mation concerning ways and means of 
furthering good teaching practices. It 
would be a mistake, however, to as- 
sume that educational standards and 
measurement standards are unrelated. 
To the extent that the results of our 
measurement are used in education, 
we run grave risks if we sacrifice 
standards of one in order to further 
standards of the other. Any research 
on a test’s influence on instruction 
should take the test’s measurement 
characteristics into account. 

A third issue is the use which will 
be made of the ccr scores at schools 
and colleges. If one accepts the inter- 
pretation of available evidence, this 
test is one of moderately high validity 
but low reliability. This combination 
of measurement characteristics 
sharply limits the use of the scores 
with an individual student. We can be 
reasonably sure that any group statis- 


tics, such as a class average, will be a 
dependable measure of group perform- 
ance on the test, since these statistics 
are less sensitive to unreliability than 
are the individual scores. We cannot, 
however, treat the scores of individual 
students with the same degree of con- 
fidence since these bear the brunt of 
any unreliability in the test. The an- 
alogy here is with group statistics of 
the incidence of a major disease. Med- 
ical science can predict with consid- 
erable accuracy the incidence of polio 
or heart disease in large segments of 
the population. It cannot predict with 
any certainty whether an individual 
will contract the disease. For this rea- 
son, we believe that any future admin- 
istration of this test in the Board’s 
regular series should be accompanied 
by adequate safeguards to minimize 
any possible misuse of the scores. 

And finally, even though we are 
questioning the effectiveness of the GcT 
as a measure of writing ability (or per- 
haps because of this), we recognize 
the urgency to continue the research 
efforts of teachers and measurement 
people alike. We do not know whether 
progress will be achieved by refine- 
ments of this test or whether new lines 
of inquiry will be more fruitful. We 
do know that the Board and its com- 
mittees will continue to give careful 
consideration to the question of mea- 
surement of writing abilitv. 


The test satisfies an educational need 


BY EARLE G. ELEY 


The present status of the General Com- 
position Test is the result of coopera- 
tive endeavor. Since I became associ- 
ated with this test some four years ago, 
I have seen it develop through the 
efforts of 40 or 50 teachers of English 
and through the serious contributions 
of a great many more people who, 
though not teachers of English, are 
devoted to the idea that writing should 
be tested by writing. 

There was a time when no one in his 
right mind would have dreamed of 
suggesting that the ability to write be 
tested by any means other than by hav- 


ing the individual write. But the time 
came during the twenties and thirties 
when educational psychologists began 
to experiment with indirect measures 
of writing made up of objectively 
scored items which they hoped would 
test validly the ability to write. This 
experimentation arose from the con- 
cern about the inconsistency, or more 
technically, unreliability, with which 
readers were found to be able to grade 
essays. 

As early as 1912 Starch and Elliott 
pointed out that common school teach- 
ers disagreed in their scoring of the 








same essay by as many as 35 to 40 per- 
centage points.’ These findings were 
reproduced over and over by other 
investigators during the twenties. The 
lack of objectivity in scoring essays 
constituted a major problem, since 
most testing at that time was done by 
essay. It is not difficult to understand 
that, given the temper of American 
educational psychology of the period, 
the efforts to solve this problem should 
have been directed not to the improve- 
ment of essay tests, but to the develop- 
ment of objectively scored tests of 
writing competence. The substitution 
of such “new-type” tests for the essay 
was argued, not on the grounds of 
validity, but rather on the grounds 
that essays, however valid, could not 
be read reliably. The objective tests, 
it was felt, not only could achieve ex- 
tremely high statistical reliability, but 
also were economical to administer, 
and provided a solution to the problem 
of growing numbers of candidates. 
The pioneers of this movement vali- 
dated their tests by going to the actual 
human operations which they intended 
to measure. Thus, in the case of writ- 
ing, they went to essays. 


Product vs. process 


It is ironical, but understandable, 
that the objective test movement 
should have developed, by now, so in- 
tricate a maze of testing procedures 
that frequently at least a part of the 
validation of one objectively scored 
test is made by judging it against other 
objectively scored tests. This proce- 
dure may be illustrated historically by 
the development of intelligence tests. 
Mr. Pearson’s article, our companion 
on these pages, affords an example of 
this modern practice: the criterion for 
writing which seems to be most highly 
regarded is the Verbal score of the 
Scholastic Aptitude Test, an objec- 
tively scored measure. 

The argument of Mr. Pearson’s 
article is a comparatively simple one. 
It begins with a description of the GcT 
and then proceeds to an analysis of 
the test’s reliability and validity. The 
conclusion drawn is that the GcT is not 
a satisfactory measure of writing com- 





*Daniel Starch and Edward C. Elliott, 
“Reliability of the High-School Work in 
English,” School Review, September, 1912, 
pp. 442-457. 
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petence. Now the problems of reader 
reliability and validity posed by the 
GcT are no different in nature from 
those posed by any free essay. By 
structuring the essay question to create 
a common starting point for both writ- 
ers and readers, and by objectifying 
some of the elements that go into 
reader judgment, the ccT has surely 
contributed to a solution of the prob- 
lems of the essay test. Yet actually the 
Pearson report seems finally to con- 
clude that: no essay test is likely ever 
to be read with sufficient reliability 
nor is an essay a valid test of writing 
in the first place. 

It is unlikely that many teachers 
will concur in this conclusion. Yet it 
is an understandable one, and is surely 
a logical outcome once the testing of 
educational objectives fell into the 
hands of educational technicians. 
These technicians, originally seeking 
only to serve the classroom teachers, 
have by now come to dominate the 
classroom with the complexity and the 
importance of their functions. Their 
services to the teacher have been im- 
mense, yet applied techniques fre- 
quently lag behind the theory upon 
which they should be based and the 
mechanistic views of human psychol- 
ogy which prevailed during the thir- 
ties too often remain the basis of pres- 
ent-day practice. Writing is a creative 
process and the products of writing 
never have lent themselves readily to 
a mechanistic theory. They still do not, 
and when statistical analysis is applied 
to essay products without a regard for 
the difference between the results of 
an essay test and the results of an ob- 
jective test, some confusion is likely 
to arise. 

In recent years the tenor of educa- 
tional psychology has changed con- 
siderably. The human organism is now 
regarded as a dynamic and creative 
whole rather than as simply the sum 
of a series of mechanistically defined 
parts. This shift in viewpoint has im- 
plications for language behavior. The 
mechanist concerns himself with the 
language product (and indeed, this 
has been and for the most part re- 
mains, the emphasis of the profes- 
sional linguist). In keeping with the 
modern view of human psychology, it 
seems more feasible to concern one- 
self with the linguistic process. These 
observations are relevant to the grad- 


ing of essays. It has been customary in 
the past to rank the essay products 
from Excellent to Nauseating without 
regard to what that product reveals 
about the writer. One of the departures 
in theory underlying the scoring of the 
ccT has been the insistence that the 
reader use the essay as a body of evi- 
dence for judging the writing process 
and hence that he attempt a judgment 
about the candidate’s habitual writing 
behavior rather than attempting sim- 
ply to rank the essays themselves. 
From this premise it followed logically 
that, since writing is a complex syn- 
drome of human behaviors, the judg- 
ment of the essay should be broken 
down into separate qualities: Mechan- 
ics, Style, Organization, Reasoning, 
and Content, in the case of the ccr. 


Reliability reviewed 


It was felt that this kind of approach 
would lead to an increased objectivity 
of scoring and would result in a more 
meaningful report than would a single 
grade. The results of the two experi- 
mental readings of the cct at Andover 
and Lawrenceville and of its first of- 
ficial reading at Haverford seem to 
bear out this assumption. If the judg- 
ments of readers are grouped so as to 
measure areas of agreement against 
areas of disagreement, the reader re- 
liability coefficients at the three con- 
ferences turn out to be as follows: 


Correlation of First Reader vs. 


Second Reader at Three ect 
Reading Conferences 

M S O R Cc 
Andover 94 88 86 90 .92 
Lawrenceville 85 .90 87 .90  .93 
Haverford* A 2 8&6 #8. & 





*Based on the sample of 370 cases ran- 
domly selected by Dr. Swineford in her 
analysis of the ccr reading; see Mr. Pear- 
son’s report, page 5, and “A question of re- 
liability,” page 12. 


The reliability coefficients reported 
above do not reflect the influence of 
third readings, which were required 
whenever the first and second readers 
disagreed beyond one grade level. As 
Mr. Pearson has pointed out, the 
readers used border marks to indicate 
highness and lowness within a level as 
well as a mark to indicate that an essay 
was to be placed squarely within the 
grade level. If one were to look at 
reader reliability in another way, ig- 


~ 


noring the adjustments that were made 
when first and second reader data were 
grouped into areas of agreement and 
disagreement, but adding the influence 
of the third reader, thus reflecting in 
the reliability figures what actually 
went into the final grade, though bas- 
ing these figures rigorously upon the 
“raw data,” the following reliability 
coefficients result: Mechanics, .88; 
Style, .85; Organization, .82; Reason- 
ing, .83; and Content, .83.° 


The four and ten-point scale 


Eighteen years ago John M. Stal- 
naker, who perhaps more than anyone 
else in the thirties worked for the 
essay test, and from whose pioneering 
efforts many of the features of the ccT 
were derived, came to the conclusion 
that if the scoring of an essay was to 
be considered satisfactory, “a reader 
reliability of around .90 is desirable.””* 
The ccr has approached so closely the 
goal mentioned here by Mr. Stalnaker 
that we may probably consider the 
problem of reader reliability as solva- 
ble. It seems very likely that on the 
basis of the experience gained at 
Haverford and improved techniques 
of training, abetted by improved tech- 
niques of administration of the con- 
ference (to wit, the novel use of IBM 
equipment), reader reliability of the 
ccT can be improved up to a point. It 
should be noted, however, that the 
reading of essays is not precisely anal- 
ogous to the comparison of samples 
of iron run through an electric fur- 
nace. Whereas in the latter case, cor- 
relations below .98 might be regarded 
as disastrous; in the former, correla- 
tions above .90 should be viewed with 
astonishment, and perhaps with sus- 
picion. 

One of the assumptions underlying 
the ccT is that in so complex an area 
readers cannot make extremely precise 
judgments, and it is for this reason 
that a four-point scale was adopted for 





*These figures are based upon a random 
sample of 200 cases from the Lawrenceville 
conference, since all of the raw data from 
the Haverford conference were not available 
to the writer. However, it seems reasonable 
to assume that the two conferences were not 
markedly different in results since the cor- 
relation coefficients in the above table are so 
similar. 

"John M. Stalnaker, “Question VI — the 
Essay,” English Journal, February, 1937, p. 
134. 
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The human organism is not a series of 
mechanistically defined parts 


each of the five qualities. The border 
indications allowed the readers have 
never been considered “real” grades, 
but rather indications of where the 
reader feels the student stands on a 
continuum of good to bad. cct grades 
have never been reported except in 
terms of a four-point scale. An analy- 
sis of reader disagreements based 
upon a 10-point scale, as reported in 
Mr. Pearson’s article, is extremely 
useful in assessing the actual work of 
individual readers at a reading con- 
ference, but may be misleading if used 
as a measure of the extent of agree- 
ment. I do not wish to enter into a dis- 
cussion of what statistical technique 
is appropriate to various kinds of data. 
Possibly the simplest way to express 
the facts of the case is to point out that 
reader agreement at the two experi- 
mental conferences, before a third 
reading had been added, was about 
80 per cent and at Haverford, where 
we were faced with the unforeseen 
situation of five tables of readers and 
about 4,300 essays (that is 8,600 read- 
ings) reader agreement dropped to 
about 76 per cent. One may treat these 
percentage distributions in various 
ways, and it is apparent that different 
results may be attained. The percen- 
tages remain constant. 


The question of validity 


An adequate essay test of writing is 
valid by definition; that is to say, it 
has face validity since it requires the 
candidate to perform the actual be- 
havior which is being measured. It 
would be wrong, however, to assume 
that one essay test is as valid as any 


other since it is necessary, in order to 
achieve maximal validity in an essay, 
to secure a sample of the student’s 
optimal writing behavior. For ex- 
ample, a test which poses an impos- 
sibly difficult problem will handicap, 
or even paralyze, many candidates; at 
the other extreme, a test which poses 
too simple or too concrete a problem 
may enmesh students, and sometimes 
the most able, in a simple-minded dis- 
cussion which by no means reflects 
their true abilities. Considerable care 
was taken during the early days of the 
GCT to find means of eliciting an essay 
product which would be as valid as 
possible. 

The readings which accompany the 
essay question were designed to ac- 
complish two purposes: (1) to stimu- 
late candidates to write, regardless of 
the level of their ability, and (2) to 
minimize among candidates differ- 
ences in educational and personal ex- 
perience. Moreover, we recognized 
that the way in which an essay is 
scored is relevant to its validity. Since 
we conceived that writing is a complex 
rather than a simple behavior, we felt 
that scoring the essays according to 
the five qualities would increase, not 
only the objectivity of reading, but 
also the validity of the reported scores. 
Further, the emphasis upon viewing 
each essay as evidence of a process 
rather than as a product to be ranked 
was a device for improving both the 
objectivity and the validity of the test. 


Which teachers’ estimates? 


While there can be no doubt that 
the validity of the cct may be im- 
proved, Mr. Pearson’s conclusions 
about validity are surprising. These 
conclusions are based upon a study 
comparing GCT scores and teachers’ 
estimates of writing ability. Similar 
investigations were conducted during 
each of the first two GcT experiments 
at Andover and Lawrenceville. The re- 
ported results of these earlier studies 
were not much different from the re- 
sults Mr. Pearson now reports again. 
But the conclusions arrived at in this 
latter case are quite different from the 
conclusions which I drew from the 
earlier investigations. 

The problem of validating an essay 
test is like pulling oneself up by one’s 
own bootstraps, since the writing of an 








essay is the only direct measure of 
writing. When teachers’ estimates of 
writing ability are used as a criterion 
against which to judge the GcT, we set 
up a situation like the following. In 
one case (the GcT), a group of highly 
qualified teachers of English meet to- 
gether and, in training sessions, dis- 
cuss and come to agreement about the 
meaning of the five qualities and of the 
four levels of scoring. They then pro- 
ceed in a controlled situation to judge 
the essays. In the second case, a group 
of teachers of English, equally highly 
qualified, but working in separate 
schools, are asked to judge the writing 
competence of students, not in their 
usual way, but in the way it is done 
at the GcT reading conferences; that is 
to say, according to the five qualities 
and by the four levels, the meanings of 
which they may be expected to under- 
stand only in terms of their own ex- 
perience and in terms of their own par- 
ticular school situation. 

It will not surprise many teachers 
that a low correspondence has been 
discovered consistently between GCT 
scores and teachers’ estimates secured 
under these circumstances. I imagine, 
however, that the conclusion of Mr. 
Pearson that a carefully constructed 
essay test, painstakingly scored by 
competent teachers of English, is not 
valid will be surprising to many. The 
conclusion I have come to, after study- 
ing the results of these three similar 
investigations, is that the procedure of 
bringing English teachers together and 
allowing them to discuss what they 
mean by the objectives of English in- 
struction is an extremely valuable one 
and, in a short time, leads to a remark- 
able consensus. 


The sum of the parts 


However, Mr. Pearson’s conclusions 
with respect to validity add one sur- 
prise upon another. For example, we 
find him saying that the essays written 
in response to the GcT may test ability 
in Mechanics and Style, but do not test 
ability in Organization, Reasoning, 
and Content. As a matter of fact, we 
have not been able to produce a test 
which will allow the student to write 
an essay in which he exhibits his com- 
petence in mechanics and style, but 
organizes nothing, makes points with- 
out reasoning, and produces about a 
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A question of reliability 


GRADING SHEET 
No. of Reader___—— 
Time Finished____—_ 


No. of Paper__._._.— 


Date. s-—w Time Begun __ 


Content 


Place papers in categories 1 to 4 with an X; border may be marked 
only a or Y. 


GCT grading sheet illustrates case described below in which two readers 
assign different scores in Mechanics category. In practice, each reader uses 
a separate grading sheet and indicates scores in all five categories 


The startling difference between the ccT reader reliability correlations 
reported in their articles by Mr. Pearson (page 5) and Dr. Eley 
(page 10) arises, among other things, from a difference between the 
authors as to what constitutes “agreement” between readers. 

In the oversimplified illustration one reader scores the essay “4” in 
Mechanics and a second reader indicates by the carat (A) in the 
borderline space that he considers it a “high 3” in Mechanics. Dr. 
Eley accepts this as agreement and before computing his correlation 
for Mechanics, plots it as an instance of reader agreement, as indicated 
by the cross mark in the shaded area below. Mr. Pearson considers 
the “4” and “high 3” scores to be a case of reader disagreement and 
plots his mark in the unshaded area to indicate this. 

Since the correlations obtained are essentially numerical expressions 
of agreement, the correlations reported by Dr. Eley are higher than 
those of Mr. Pearson. 


Second Reader Second Reader 


First Reader 
First Reader 


Dr. Eley’s plot 


Mr. Pearson’s plot 





thousand words devoid of content. 
Actually, the Pearson report is based 
upon averaged coefficients secured 
from 11 different schools. In the And- 
over experiment I found about the 


same results as Mr. Pearson reports 
when I averaged teachers’ estimates 
(not from 11, but from 29 secondary 
schools). The differences among the 
schools were tremendous, ranging, for 


example in Mechanics, from .16 to .82. 
If, instead of averages, I had reported 
the correlation between GcT and teach- 
ers’ estimates, selecting for each qual- 
ity the best school, the correlations 
would have been: Mechanics, .82; 
Style, .67; Organization, .72; Reason- 
ing, .60; and Content, .58. The lowest 
correlations I could have reported 
would have been: Mechanics, .16; 
Style, .41; Organization, .22; Reason- 
ing, .42; and Content, .33. 

There is considerable evidence that 
the GcT scores in Organization, Rea- 
soning, and Content have some inde- 
pendence. For example, in the And- 
over experiment it was found that the 
score in Reasoning correlated with the 
College Board Social Studies Achieve- 
ment Test at .31, while the score in 
Content correlated at .68. On the other 
hand, the GcT score in Reasoning cor- 
related with the College Board Mathe- 
matics Achievement Test at .65, where- 
as the score in Content correlated at 
only .24. Further, in all three con- 
ferences the intercorrelations among 
qualities have demonstrated reason- 
able independence, the highest ordi- 
narily being between Reasoning and 
Content at about .75. 


Language as a process 


Research in language behavior his- 
torically has been centered upon a 
concern with language products, and 
this generalization applies not only to 
the teacher and the maker of tests, but 
to the professional linguist, the soci- 
ologist, the psychologist, the anthro- 
pologist, and even to the philosopher. 


Earle G. Eley 
is examiner in English and Humanities at 





the University of Chicago and is currently 
teaching English composition there. Dr. Eley 
led the experimental work on the General 
Composition Test and has been its principal 
interpreter and spokesman. He was chief 
reader of the first regular administration of 
the test last May. 
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We can evaluate language behavior only 
in terms of the total process 


Probably, of all of these the teacher of 
writing has come closest to dealing 
with the process of language, though 
he has rarely, if ever, reported the pro- 
cedures of his classroom in theoretical 
terms. This situation is a logical con- 
sequence of the recent history of lan- 
guage studies. During the twenties, 
thirties, and early forties, when the 
prevailing psychology in the United 
States was mechanistic, that is to say, 
behavioristic, the professional linguist 
was busily engaged in the attempt to 
make an exact science from a humane 
study, and was happy enough to ex- 
clude the investigation of a psycho- 
logical basis for language behavior 
from his enterprise. It is not surpris- 
ing, then, that whatever statement 
about the psychological basis of hu- 
man behavior we find among the 
linguists of this period, turns out to be 
a direct and naive reflection of stimu- 
lus-response psychology.’ 

Thus, we have the science of lin- 
guistics depending upon another sci- 
ence, psychology, for its theoretical 
basis. The test maker who engaged in 
constructing tests of writing ability 
was doubly handicapped in that, nur- 
tured by behavioristic psycholog,, he 
found that the professional linguist 
afforded him no more than a reflection 
of the same climate. An applied scien- 
tist, he had little to apply. That he 
should have failed to recognize that 
language behavior is actually an in- 
tegral part of the total effort of each 
personality to realize itself and is, even 
at its lowest levels, more closely re- 
lated to creative endeavor than to 
conditioned activity, is pardonable — 
but an error. 

Few modern psychologists would 
accept the label mechanistic, or be- 
havioristic. The notion that some 


*See, for example, Leonard Bloomfield, 
Language. (New York: Henry Holt and Co., 
1933), pp. 31-32. 


kinds of human behavior are complex 
syndromes of activity which represent 
wholes that are more than the sum of 
their parts is today commonly ac- 
cepted. Language clearly is such a 
human activity. When we investigate 
or attempt to evaluate language be- 
havior, we can do it only in terms of 
the total process. Were it possible to 
reduce these complex and mysterious 
aspects of the human personality to 
elements capable of exact measure- 
ment, our universe would be a simple, 
secure and some may think, a happy 
one. Too often, we have achieved pre- 
cision in the measurement of human 
behavior by mechanizing the elements 
of that behavior. Such a procedure is 
effective only part of the time: the 
creative or dynamic aspects of the 
human personality resist atomization. 
They must be assessed as wholes or not 
at all. 


Objectives of the test 


Thus, it seems unlikely that objec- 
tively scored tests can be developed to 
measure validly the ability to write, 
which is a creative aspect of the human 
personality. The Gct attempts to meas- 
ure this ability as a totality. We sought 
to develop as much precision as pos- 
sible, but we knew from the beginning 
that writing cannot be measured with 
as much precision as can, for example, 
the reaction time of knee jerks. More- 
over, we have avoided deluding our- 
selves into mechanizing the testing or 
reading procedures — a device of most 
objective measures of creative activity 
—in order to produce the happy se- 
curity of spuriously high statistics. 
The cct ought to be judged, I should 
think, in terms of whether (1) it satis- 
fies a need in American testing, (2) it 
satisfies a demand expressed by teach- 
ers of English, and (3) it achieves a 
degree of objectivity appropriate to 
the kind of instrument that it is. There 
is no doubt that from some points of 
view, we could find reasons for aban- 
doning forever the essay test. Unfor- 
tunately, the corollary to this action is 
the decline in the teaching of the art of 
writing. This is a serious consequence 
both for the individual and for our 
society. The ability to use language 
creatively is one of the last strongholds 
of the individual against a growing 
tendency toward conformity. 








School visiting made easier—and better 





BY MARY E, CHASE 





By working together college representatives can reach more 


schools, inform more students, eliminate duplication 


“And now girls, Miss Wellesley will 
talk to you about liberal arts colleges 
for women.” This introduction, made 
several years ago by the principal of 
an eminently respectable school, un- 
nerved me considerably, for I had al- 
ways pictured Miss Anyplace as a cur- 
vaceous tidbit, scantily clad in the bar- 
est-of-essential bathing suits, who 
walked rather than talked, and I was 
obviously miscast. It later developed 
that the principal in question was so 
completely startled when, in accepting 
her invitation to speak to the women 
students, I had suggested that I would 
prefer to talk about other colleges as 
well as about the one which employed 
me, that she forgot my name and re- 
membered only the college. 

I doubt that this could happen to- 
day, for there is a general acceptance 
among representatives of women’s col- 
leges and a growing recognition on the 
part of school people that what is good 
for one college is usually good for an- 
other, and that a high degree of co- 
operation with one another is not only 
possible but extremely desirable. 

School visiting by college represent- 
atives still has room for vast improve- 
ment and refinement. We shall prob- 
ably have always with us, like the poor, 
such extremes as what is described by 
an acquaintance of mine as “the oc- 
casional foray made into the field by 
last year’s football hero,” the dashing 
Beau Brummell in his Buick who 
charms the family and paints enchant- 
ing pictures of “living happily ever 
after” to the little girl whose social ma- 
turity has been slow and painful, and 
the erudite physics professor with 
baited hook who has been torn from 
his atomic pile and thrown into an un- 
familiar world by the president of his 


college, charged with the responsibil- 
ity of increasing the number of majors 
in physics at Siwashette. 

Even the normal (if there is one) 
college representative with beaten-up 
suitcase and bulging brief case, weary 
of trains, dismal hotel rooms and oc- 
casional orgies of creamed chicken 
and peas (if sufficient alumni can be 
prevailed upon to gather to meet him) 
meets some extremes in the schools. 
There is the stony glare of the grim 
headmistress who plants herself firmly 
in the front hall (if the college visitor 
has succeeded in getting that far) and 
says, “all our girls have plans,” but 
there is also the slap-happy principal 
who upon the arrival of the college 
visitor dismisses classes and asks the 
college visitor to address the junior 
high school and senior high school 
boys and girls on “Education.” Some 
school people are buried in an unend- 
ing mass of administrative detail and 





A high degree of cooperation is desirable 


can quote the exact number of square 
inches of floor space in the gymnasium 
and the number of paper towels used 
in the boys’ washroom per week, but 
throw up their hands in despair if one 
asks for information about their grad- 
uates in college or the content of the 
senior English course. Most of us in 
the business have seen them all at one 
time or another. 


Reasons for visiting 


The purpose of the school visit is, of 
course, twofold—give and take. The 
visitor secures information about the 
school which helps him to interpret 
the records of future candidates for 
his college trained in the school, and 
provided he is skillful, the college vis- 
itor gives the high school principal, 
counselor or dean, an up-to-date pic- 
ture of his institution and an under- 
standing of what is considered a well- 
qualified candidate. The school visit 
by the college representative provides 
an excellent opportunity for a face-to- 
face discussion of individual candi- 
dates. 

While an optimum arrangement 
from one point of view would be an 
annual visit to each school, it is ob- 
viously impossible for most colleges to 
support a sufficiently large profes- 
sional staff to make this possible. 
(School principals already harrassed 
by flocks of college visitors take com- 
fort! Most colleges cannot afford to 
expand their admission staffs. ) 

The compromise arrangement 
adopted by a number of colleges is to 
set up a long-term travel plan. The col- 
lege representative works out a sched- 
ule of travel to a different section of 
the country each year, so that the 
major portion of the country from 
which the college draws its students 
can be visited once in every three to 
five years. The long term travel is sup- 


plemented by annual visits to individ- 
ual schools or areas which present a 
substantial number of candidates each 
year and which might be referred to 
as “feeder areas” or “feeder schools.” 

With the vastly increased numbers 
of applications anticipated in the next 
few years and with applications which 
will come in many cases from new 
schools, the necessity for a continua- 
tion and extension of school visits is 
obviously essential. Yet at the same 
time, if colleges are to maintain the 
quality of their academic programs, 
they are faced with the continuing 
need for economies and cannot in most 
instances afford to increase their ad- 
mission staff proportionally. Is it pre- 
sumptuous to suggest that a college 
desires to be represented by a well- 
informed and well-trained person— 
preferably one who has had previous 
teaching experience in either the sec- 
ondary or collegiate field or both, and 
that such individuals are not easy to 
find nor are they inexpensive ? 

Each college which operates a con- 
sistent program of school visiting has 
in its possession a considerable amount 
of information about the various sec- 
ondary schools which have been vis- 
ited. The gathering of this information 
by colleges across the country is very 
expensive, especially when one con- 
siders that a single college visitor could 
easily make copies of his information 
about a given school or schools and 
send it to five colleges, let us say, at 
considerably less effort, energy, and 
expense than is currently required 
when five college visitors secure the 
same information separately for their 
records, 

Perhaps the emergency presented 
by these two phenomena, increased de- 
mands on the admissions officers and 
duplication of information and effort, 
points as often only an emergency 
can, to a very simple and considerably 
more efficient method of handling this 
whole problem. 


A cooperative effort 


Is it too much to expect Miss Welles- 
ley, Mr. Kenyon, or Mr. and Mrs. Po- 
mona to be informed about other col- 
leges of their type so that they can rep- 
resent more than their own particular 
institution? Surely no one person 
could be expected to have complete 
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The purpose of the school visit is twofold 


factual information on all colleges, but 
there is no reason to believe that a pro- 
fessional in one college cannot repre- 
sent similar sister or brother institu- 
tions. I am not proposing, for example, 
a College Board Representative to rep- 
resent College Board member colleges, 
but rather the formation of small in- 
formal groups of colleges which have 
common programs and interests and 
the desire to cooperate with one an- 
other in the presentation of their pro- 
grams to schools and to candidates in 
the schools. 

It is important that such college 
groups be kept small, since too large 
and too diverse a group will rob each 
institution of its individuality. Unless 
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executive vice president and director of 


Mary E. Chase, 


admission of Wellesley College, is particu- 
larly interested in the college’s relations 
with prospective students, schools, and alum- 
nae. She has held school and college posts 
and served in the waves during World War 
II before going to Wellesley, in 1946, as 
director of admission. 


the group is small and reasonably ho- 
mogeneous, it is quite impossible for a 
single representative of any one of the 
colleges to be adequately informed, 
and his usefulness is thereby reduced. 
It is important that such a grouping 
be kept informal, for too much organi- 
zation can defeat the very purpose of 
the cooperative venture, and the values 
to be achieved can be smothered by 
endless red tape and a sheeplike con- 
formity. Each cooperating college in 
the group must maintain its own integ- 
rity and be allowed “to be itself.” 
How would such a plan work? When 
several colleges recognize themselves 
as a natural group, it would be com- 
paratively simple to determine in what 
area, or areas, of the country school 
visits are most essential, and by co- 
operative means a master travel plan 
could be worked out so that not more 
than one representative would visit the 
schools in a particular area during a 
given academic year. If, for example, 
there were three colleges in the group, 
the coverage would be tripled. It 
would be a comparatively simple mat- 
ter for the group to work out jointly in 
advance some agreement as to the 
type of information desired about each 
school, so that all members of the 
group could share it. It would be 
equally important for the colleges to 
work out together, in advance, policies 
with respect to answers to questions 
regarding unusual programs of sec- 
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ondary school study, exceptions to re- 
quirements, and specific problems re- 
lating to each individual college. The 
important similarities of the colleges 
should be familiar to all members of 
the group, as should important dissim- 
ilarities. One should be able to present 
the colleges directly to counselors and 
students, referring specific questions 
to individual catalogues or individual 
offices. 

While the actual geographical loca- 
tion of the colleges in any given group 
would be much less important than the 
internal similarities of the colleges in 
the group, there would be certain ad- 
vantages to be gained if the colleges 
were located in different parts of the 
country. An ideal grouping might be a 
southern college, a New England col- 
lege, a midwestern college and a far 
western college. There would be obvi- 
ous advantages to each college in the 
group in acquainting students from 
widely diverse geographical areas 
about the programs in each college. 


The time is now 


There are those who would argue 
that since the colleges have a seller’s 
market, why visit at all? Why recruit 
students when application lists are al- 
ready bursting the files? The reason is 
that school visits are not recruiting 
ventures, but rather attempts to im- 
prove understanding and communica- 
tion between schools and colleges. 
From the college’s point of view, 
school visiting is most important when 
the supply of qualified candidates ex- 
ceeds the number of places available 
to them, for it is at this time that 
rumors, misunderstanding, and folk- 
lore are at their height, and the time 
when it is essential that there be close 
understanding between the schools and 
colleges and open lines of communica- 
tion. 

It is hoped that, in the future, school 
systems will recognize and adopt pro- 
grams similar to the one currently in 
operation in Atlanta, Georgia, where a 
forward-looking administration has 
appointed an experienced school man 
as a liaison person between the Board 
of Education and the colleges attended 
by Atlanta students. This liaison officer 
is the authorized representative not of 
one school, but of all the public high 


schools in the county, and has close 
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Too much organization can defeat the very 
purpose of the cooperative venture 


association with students already in 
college and with the principals and 
guidance officers in each school. Ad- 
vance notice to the liaison officer of the 
arrival in Atlanta of a representative 
of one of the “groups” described above 
would facilitate an economical use of 
the time of both the college representa- 
tive and guidance personnel, since a 
single, rather than separate meetings 
of several representatives, could be 
easily arranged and many problems 
and questions could be aired in a rela- 
tively short time to the advantage of 
the already overburdened counselors. 

It is conceivable that a cooperative 
effort among the admissions represent- 
atives might result in a similar effort 
among their alumni to search out and 
acquaint talented young people with 
the opportunities available to them. In 
spite of swollen application lists, prob- 
ably no college ever has too many good 
candidates. What better source of in- 
formation is there about these good 
candidates, outside of the school au- 
thorities themselves, than educated 
members of the community? Alumni 


Cooperate! 


Recruiting is a naughty word. 
So’s competition I have heard. 


Let’s group ourselves across 
the nation, 


And try, for once, cooperation. 





who are loyal to their college and wish 
to “do something” for it could be in- 
vited to participate in a program 
which does not ask them for money. 
Many young alumni just starting out 
in business and professions and unable 
to assist their college financially might 
contribute generously to such a co- 
operative effort. The colleges would 
benefit by their participation both di- 
rectly in terms of students, and indi- 
rectly in terms of interest. 


Additional benefits 


Such a grouping of colleges might 
have other very valuable side results. 
One of these might be the lifting of 
the veil of secrecy and the elimination 
of the atmosphere of hocus-pocus 
which quite unintentionally surrounds 
the admissions process in many col- 
leges. In order to discuss the admis- 
sions policies of the colleges in a given 
group, one must know what their poli- 
cies are—which might encourage some 
admissions committees to collect their 
thoughts and express them on paper. 
Differences in procedure in individual 
colleges in the group would come to 
light, and one college might profit 
from the experiences of another. The 
whole area of requirements, too, would 
come under scrutiny and enable the 
institutions in the group to pool their 
studies and have a better basis upon 
which to determine the actual courses 
essential for admission. 

This proposal would seem to deny 
the maxim, “Never urge people to do 
together what the self-reliant among 
them can do better alone” but in the 
present and future emergency in col- 
leges, each college in such a group 
could profit from the union and could 
at the same time be allowed to retain 
its individuality. 

There are dangers, of course, but 
not the same dangers which business 
or industry would encounter in such a 
grouping. Certainly each college in 
such a group must be constantly on 
guard against mass anonymity and ex- 
cessive standardization. But as long as 
faculty members of colleges act as in- 
dividuals and insist that their colleges 
“be themselves” the danger of en- 
croachment by the monster of Big Edu- 
cation is not great. 

The plan certainly seems worth try- 


ing. 


How to “get by” written examinations 


BY STEPHEN LEACOCK 


Sir: You are kind enough to refer to 
certain of my writings in regard to the 
difficulties and fallacies of written ex- 
aminations. You ask me if there is any 
way—if I have your phrase right—to 
“get by.” I think there is. 

Every student should train himself 
to be like the conjurer Houdini. Tie 
him as you would, lock him in as you 
might, he got loose. A student should 
acquire this looseness. 

For the rudiments of education, 
there is no way round. The multiplica- 
tion table has got to be learned. They 
say Abraham Lincoln knew it all. So, 
too, the parts of speech must be com- 
mitted to memory, and left there. The 
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The multiplication table has to be learned 


names of the Wessex Kings from Al- 
fred (better Aelfoydd) to his Danish 
successor Half-Knut should be learned 
and carefully distinguished from the 
branches of the Amazon. 

But these rudiments once passed, 
education gets easier and easier as it 
goes on. When one reaches the stage 
of being what is called a ripe scholar, 
it is so easy as to verge on imbecility. 

Now for college examinations, once 
the student is let into college, there are 
a great number of methods of evasion. 
Much can always be done by sheer il- 
legibility of handwriting and by 
smearing ink all over the exam paper 
and then crumpling it up into a ball. 

But apart from this, each academic 
subject can be fought on its own 
ground, Let me give one or two ex- 
amples. 

Here, first, is the case of Latin trans- 
lation—the list of extracts from Caesar, 
Cicero, etc., the origin of each always 
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indicated by having the word Caesar, 
etc., under it. On this we seize as our 
opportunity. The student doesn’t need 
to know one word of Latin. He learns 
by heart a piece of translated Caesar, 
selecting a typical extract, and he 
writes that down. The examiner merely 
sees a faultless piece of translation and 
notices nothing—or at least thinks that 
the candidate was given the wrong ex- 
tract. He lets him pass. 

Here is the piece of Caesar as re- 
quired: 

These things being thus this way, 
Caesar although not yet did he not 
know neither the copiousness of the 
enemy nor whether they had frumen- 
tum, having sent on Labienus with an 
impediment he himself on the first day 
before the third day, ambassadors hav- 
ing been sent to Vercingetorix, lest who 
might which, all having been done, set 
out. 

Cicero also is easily distinguished 
by the cold, biting logic of his invec- 
tive. Try this: 

How now which, what, oh Catiline, 
infected, infracted, disducted, shall 
you still perfrage us? To what expunc- 
tion shall we not subject you? To what 
bonds, to what vinculation, to how 
great ahyphen? I speak. Does he? No. 

Cicero. In (and through) Catiline 

The summation of what is called the 
liberal arts course is reached with such 
subjects as political theory, philos- 
ophy, etc. Here the air is rarer and 
clearer and vision easy. There is no 
trouble at all in circling around the ex- 
aminer at will. The best device is found 
in the use of quotations from learned 
authors of whom he has perhaps—in- 
deed, very likely—never heard, and 
the use of languages which he either 
doesn’t know or can’t read in blurred 
writing. We take for granted that the 
examiner is a conceited, pedantic man, 
as they all are—and is in a hurry to 
finish his work and get back to a 
saloon. 

Now let me illustrate. 

Here is a question from the last 





Reprinted from the “To the Editor” col- 
umns of The Daily Princetonian, Wednes- 
day, January 26, 1938. 


Princeton examination in Modern 
Philosophy. I think I have it correct 
or nearly so. 

“Discuss Descartes’ proposition, 
‘Cogito ergo sum,’ as a valid basis of 
epistemology.” 

Answer: 

“Something of the apparent origi- 
nality of Descartes’ dictum, ‘cogito 
ergo sum,’ disappears when we recall 
that long before him Globulus has writ- 
ten “Testudo ergo crepito,’ and the 
great Arab scholar, Alhelallover, writ- 
ing about 200 Fahrenheit, has said, 
‘Indigo ergo gum.’ But we have only to 
turn to Descartes’ own brilliant con- 
temporary, the Abbé Paté de Foie 
Gras, to find him writing, ‘Dimanche, 
lundi, mardi, mercredi, jeudi, ven- 
dredi, samedi,’ which means as much, 
or more, than Descartes’ assertion. It 
is quite likely that the Abbé was him- 
self acquainted with the words of 
Pretzel, Wiener Schnitzel and 
Schmierkase; even more likely still he 
knew the treatise of the low German, 
Fisch von Gestern, who had already set 
together a definite system or scheme. 
He writes: “Wo ist mein Bruder? Er 
ist in dem Hause. Habe ich den Vogel 
gesehen? Dies ist ein gutes Messer. 
Holen Sie Karl und Fritz und wir 





Fight each subject on its own ground 


werden alle ins Theater gehen. Danke 
Bestens.’ ” 

There, one can see how easy it is. I 
know it from my own experience. I re- 
member in my fourth year in Toronto 
(1891) going into the exam room and 
picking up a paper which I carelessly 
took for English philology; I wrote on 
it, passed on it and was pleasantly sur- 
prised two weeks later when they gave 
me a degree in Ethnology. I had an- 
swered the wrong paper. This story, 
oddly enough, is true. 








The case against freshman flunk-outs 








How many applications for admission 
are needed to enable a college to select 
a class of certain size which meets a 
certain standard of scholastic per- 
formance? 

While this question cannot be an- 
swered precisely and to the man, it 
was found at Case Institute of Tech- 
nology that predictions can be made 
which are very useful administratively. 
A good understanding of the way a 
college’s selection criteria actually 
operate may show, for example, that 
only 500 rather than 750 applications 
need be evaluated to obtain the desired 
class—and keep it. A difference of this 
kind may also be considered somewhat 
more than statistically significant 
when translated in terms of the size of 
an admissions office staff and its work 
load. 

The basic choice the college has is 
whether to establish its admissions 
goals in expectation of losing a certain 
percentage of students through failure, 
or to employ predictive procedures 
which identify those most likely to suc- 
ceed academically. In the first case the 
emphasis is on numbers and the goal 
is a freshman class large enough to in- 
clude that certain percentage of stu- 
dents whose departure later will be ac- 
cepted as natural and normal and char- 
acterized as “attrition.” The more se- 
lective method, however, not only lo- 
cates those individuals whose survival 
in college is likely, but finds more of 
them in any given number of appli- 
cants. 

The Case Institute experience sug- 
gests that although attrition may be a 
necessary evil, selective measures en- 
able a college to plan its recruitment 
and admissions activity much more 
wisely. An institution can know with 
very little doubt how many applica- 
tions will be needed and how much 
effort will be required to get them in 
order to produce a given number of 
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acceptable applicants. It can also know 
how the attrition rate would vary if 
the admissions standards were 
changed to produce a larger or smaller 
entering class. 

Last year at Case, for example, 25% 
of the freshman class were separated 
for reason of poor scholastic perfor- 
mance or withdrew voluntarily, most 
of them also for academic reasons. 
How would selection by an appropri- 
ate battery of College Board tests, or 
by a group of tests which we can iden- 
tify for the present as the Case Battery, 
or by high school rank in class, influ- 
ence that attrition rate? Let us con- 
sider the accompanying table which 
shows the rate of attrition expressed as 
a cumulative percentage of the fresh- 
man class grouped by deciles accord- 
ing to grade point averages predicted 
on the basis of the three criteria men- 
tioned above.” 

For purposes of illustration, assume 
that Case wished to cut the total fresh- 
man attrition rate in half. Assume also 
that the 370 students who made up the 
freshman class constituted instead a 
group of applicants from which we 
were ultimately to admit a class. The 
table shows that an attrition rate of 
12% existed in the first six deciles of 
the class when it was ranked according 
to point averages predicted by the Col- 
lege Board battery. The same attrition, 
12%, is found in the first five deciles 
when the class was ranked according 
to point averages predicted by the Case 
Battery. The 12% is found in the first 
four deciles when the ranking was by 
point averages predicted by high 
school rank in class. 

This means that the College Board 
tests would have selected a class of 





*The statistical analyses for this study 
were done by Dr. Fred C. Leone, assistant 
professor of mathematics, Case Institute of 
Technology. 


BY JOHN R. VALLEY 


222 students out of the 370 applicants, 
the rate of attrition being 12%. The 
Case Battery would have selected 185, 
and the rank in class criterion 148 stu- 
dents, for the same attrition. 

Thus, selection by means of the 
Board tests would have given Case the 
opportunity to admit 74 more stu- 
dents into a class with a 12% attrition 
than could have been secured by 
means of high school rank in class and 
37 more students than would have 
been found by the Case Battery. 

The advantage here lies in the ca- 
pacity of the Board tests to identify, 
within a particular group tested, a 
larger number of individuals who can 
meet a certain performance standard. 
Taking this advantage, Case would 
have fewer students to recruit, fewer 
applications to process, fewer admis- 
sions interviews, fewer high school 
transcripts to study—fewer of every- 
thing except qualified students who 
would complete their freshman year 
satisfactorily. 

In order to actually net a class of 
450 freshmen which would have an 
attrition only one-half that of a year 
ago, it is estimated that Case would 
need twice as many completed appli- 
cations with Board test results as it had 





John R. Valley 
was assistant dean of students at Case Insti- 
tute of Technology at the time the study de- 
scribed above was conducted. He left Case 
last December to become assistant project 
director for the College Board at the Edu- 
cational Testing Service. 


in September 1953. Two and one-half 
times as many processed with the Case 
Battery and three times as many from 
which selection would be made by high 
school rank in class would be needed to 
yield the 450 freshmen. 

The data on which this illustration 
is based are from the latest study to 
reflect Case’s long-standing interest in 
the prediction of scholastic success, 
which dates back to the early twenties. 
Following World War II, the Institute 
used the Pre-Engineering Inventory as 
a selection device for admission. With 
the elimination of nation-wide testing 
facilities for the Pre-Engineering In- 
ventory, we began to experiment with 
the Pre-Engineering Ability Test. 

Looking forward to the probable 
figures of the college-going population 
of the next decade, we felt that the 
need for better prediction would in- 
crease in the future. We were also 
aware of the time lag between the initi- 
ation of a research program, the at- 
tainment of meaningful results, and 
the application of useful research find- 
ings. It was obvious that any predic- 
tion study which depended on first 
year academic performance for its cri- 
terion of success would take at least a 
year before that part of the data was 
available. 

In 1953, with the cooperation of the 
College Board and the technical assist- 
ance of the Educational Testing Serv- 
ice, we began a study designed to show 
the comparative use of the predictors 
whose application to a particular ad- 
missions situation was described 
above. The Board provided a special 
campus administration of the Scholas- 
tic Aptitude Test, which unlike the 
other tests used in the experiment is 
not available in the Board’s placement 
test program.’ The Ets staff contrib- 
uted importantly to the success of the 
plan with its suggestions for the design 
and methodology of the research. 

The predictors of scholastic success 
selected for study were: 


1. High school record, as measured by 
rank in school graduation class 


*The College Board customarily supplies 
testing materials and scoring and reporting 
services at no cost to those colleges which 
wish to explore the possibility of using the 
tests as an admissions requirement and 
agree to conduct formal validity and norms 
studies at their own expense and report the 
results to the Board. 


Rate of attrition* for the freshman year (expressed as a cumulative per cent) 


Basis of predicted point average 












































College Board Case battery 
Deciles of the battery plushigh plusrank 
Number of class according school rank in class and High school 
students (37 in to predicted and recency recency rank in 
each decile) point average variables variables class 
37 1 3% 5% 7% 
74 land 2 5 5 8 
lll 1 through 3 8 11 9 
148 1 through 4 8 ll 12 
185 1 through 5 9 12 14 
222 1 through 6 12 16 17 
259 1 through 7 17 18 21 
296 1 through 8 19 20 22 
333 1 through 9 22 23 23 
370 1 through 10 25 25 25 





* Attrition = separations and withdrawals for all reasons. 


2. College Board tests (Scholastic Ap- 
titude Test and the Achievement 
Tests of Chemistry, Physics, and 
Advanced Mathematics ) 

3. The “Case Battery” which had been 
used at Case largely for post-admis- 
sion guidance and consisted of: 

a. Pre-Engineering Ability Test 

b. Cooperative General High 
School Mathematics Test, Re- 
vised Series Form O 

c. Cooperative Physics Test, Re- 
vised Series Form Y 

d. Cooperative Chemistry Test, Re- 
vised Series Form Y 

4. Lapse of time since completion of 

high school physics 

. Lapse of time since completion of 

high school chemistry 


uw 


The entire class of 370 students who 
enrolled in September 1953 took the 
College Board tests during their ori- 
entation week at the start of the first 
term. The Case Battery data were 
available for the whole group as fol- 
lows: 22% tested with the full battery 
prior to admission; 37% took the 
three achievement tests prior to admis- 
sion and the Pre-Engineering Ability 
Test during orientation week; 41% 
tested with the full battery during ori- 
entation week. In other words, 41% 
of the class was admitted exclusively 
on the basis of their secondary school 
records. Either complete or partial test 
data from the Case Battery for the re- 


maining 59% were available at the 
time they were admitted. 

In the study, we compared each of 
the principal predictors with scholas- 
tic success, which was measured in 
terms of the freshman year grade point 
averages. The correlations for each 
predictor were as follows: 


Predictor Grade correlation 


College Board tests 


SN ied cc nc oN oa Sea ewe 331 
SAT-Mathematical ............... 442 
CIES ice cota ris cavowooe 438 
WOR ohio a kPa ov sdnnkccownt 408 
Advanced Mathematics ........... 401 
Case Battery 
Pre-Engineering Ability ........... 341 
Cooperative Mathematics .......... 356 
Cooperative Chemistry ............ 349 
Cooperative Physics ............. 298 
High school data 
Rank in graduating class .......... .299 
Recency of physics study .......... 105 
Recency of chemisty study ........ .089 


This showed that the College Board 
tests did the best job of predicting and 
that the Case Battery ran just a little 
better than rank in class. The recency 
variables were revealed to have low 
correlations with first year grades. 
Using this information, as well as 
intercorrelations between the predict- 
ors, we found that a multiple correla- 
tion of .600 could be achieved by com- 
bining the Board test and high school 
data. A multiple correlation of .517 





resulted from a combination of Case 
Battery and high school data. It was 
apparent that our prediction of aca- 
demic success could be improved. 
The computation of correlation co- 
efficients also enabled us to derive 
equations for the prediction of first 
year grade point averages. Three such 
equations were derived and three pre- 
dicted point averages calculated for 
each student on the following bases: 


1. College Board test scores, together 
with high school rank in class and 
recency of completion of study of 
physics and chemistry 

2. Case Battery test scores, together 
with high school rank in class and 
recency of completion of study of 
physics and chemistry 


3. High school rank in class alone 


The freshman class was divided into 


deciles, as illustrated by the accom- 
panying table, on the basis of each of 
the predicted averages. Students in the 
top 10% of the class on the basis of 
their predicted averages were placed 
in Decile 1, those in the next highest 
10% in Decile 2, etc. The table shows 
the number of chances out of 100, com- 
puted from observed grade point aver- 
ages, of a student being in good aca- 
demic standing both semesters when 
his performance is predicted by each 
of the three point averages. 
Predictions based on College Board 
scores are shown by the table to be 
more definitive than those which de- 
pend on the other criteria of selection. 
If a student fell in the first decile ac- 
cording to his point average calculated 
by the College Board equation, for ex- 
ample, he had 97 chances in 100 of 
being successful. His chances with the 
Case Battery were 92 out of 100, and 


with the high school rank in class 88 
out of 100. The difference in degree of 
definition becomes especially note- 
worthy at lower points on the table. 
The student who fell in Decile 10 ac- 
cording to the College Board predictor, 
for example, had only 23 chances out 
of 100, while the Case Battery shows 
40 and high school rank alone 43 
chances. 

The college which wants to elimin- 
ate unpromising candidates before 
they become failing students and iden- 
tify those most likely to succeed aca- 
demically in any given number of ap- 
plicants will find this kind of appraisal 
of its admissions criteria most reward- 
ing—much more rewarding than the 
cold bewildered stare of a faculty 
member who is informed merely that 
a correlation of .45 has been found to 
exist between SAT-Mathematical and 
first term grade point averages. 


Prediction of First Year Scholastic Success at Case Institute of Technology 





According to 
point average 


Chances in 100 That Student Would Be in Good Standing Scholastically Both Semesters 
































If ranked in predicted by 10 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 
PO Vr remem en on ee Ra ieee Oa i ee eee Poy or. 
eeeesceceeeeee SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSOSOS 
ee nee Ee EE. ee ECM ce ee a Fo vg 
eeeeceoeooooe SSOSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSHSSSSOSSSOSSOECOSE 
ODD os pea ae ete Oe CA ake Cn tee ok REN, Baek Is 
eeceeeseoseeo SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSOSSEOSSEOE 
RN cia ae ee rhe PIE ae eS ee eRe ee Tb a We 
eeeceseseoseo SOSSSSSSSSSSSSSSSSSSSSSOSSSSSSSSSSSSSSSSSSSSOSSSSOSSOSESOES 
a a ae a a ee a 
eeeceoeeoesee SSHSSSSSSSSSSSSSSSSSSSSSSSHSSSSSSSSSSSSSSSSSSSSSSOSSSESSSSESES 
ee Pe ee ee ae) eee en ne a ey Semen 
eceeecoessoeoe SSCHSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSOSSSE 
OS ane De ee ee 
eeceeseeseeeee SSCOSSSSSSSSSSSSSSSSSSSSOOSSEE 
OORT irik Aaa heat See A ae aR os 
eecceseesoeee SSCSSSSSSSSSSSSSSSSSSSSSHPSSSSSSSSSSSSSSSSSSSOSE 
PROS eee een eee dae tae 
Cdececcccccce coccceccccccccocccoosecoeesooecoeceoooee 
NU ne en ee Fe a 
eeceeceeeeseeeo SSSCSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSOSSOSEE 


= CEEB Battery, High School Rank in Class and Recency of Completion of High School Physics and Chemistry. 


= Case Battery, High School Rank in Class and Recency of Completion of High School Physics and Chemistry. 


ecevcececeee — High School Rank in Class alone. 
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e Adelphi College 
Agnes Scott College 
e“ Albertus Magnus College 
e“ Alfred University 
“Allegheny College 
“Amherst College 
Antioch College 
“Bard College 
“Barnard College 
“Bates College 
“Beaver College 
Beloit College 
¢ Bennington College 
Boston College 
¢ Boston University 
“Bowdoin College 
“Brandeis University 
e“Brown University 
e“Bryn Mawr College 
¢“Bucknell University 
© Caldwell College 
¢”California Institute of Technology 
“Carleton College 
“Carnegie Institute of Technology 
¢ Catholic University of America 
¢“Cedar Crest College 
¢ Chestnut Hill College 
e“Claremont Men’s College 
e“Clark University 
“Clarkson College of Technology 
“Colby College 
“Colgate University 
© College of Mount Saint Vincent* 
¢ College of New Rochelle* 
¢ College of Notre Dame of Maryland* 
© College of Saint Elizabeth* 
¢ College of William and Mary 
College of Wooster 
¢“Columbia College 
“Connecticut College 
Cooper Union for Advancement of 
Science and Art 
“Cornell University 
“Dartmouth College 
“Davidson College 
¢“Denison University 
¢ DePauw University 
“Dickinson College 
¢ Drew University* 
¢ Duke University* 
“Dunbarton College of Holy Cross* 
e Elmira College* 
e“Emmanuel College 
Emory University 
“Fordham College 
“Franklin and Marshall College 
Georgetown University 
George Washington University 
¢ Georgian Court College* 
¢ Gettysburg College 
©“Goucher College 
Grinnell College 
“Hamilton College 
e“Harvard College 
“Haverford College 
“Hobart and William Smith Colleges 
¢“Hollins College 
¢ Hood College 
¢ Immaculata College 
e“Jackson College for Women 
Kalamazoo College 
“Kenyon College 
¢ Knox College* 
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College Board member colleges 


with special reference to participants in the College Scholarship 
Service and subscribers to the Candidates Reply Date 


Of the 162 member colleges of the Col- 
lege Board listed below, 95 are par- 
ticipating in the College Scholarship 
Service. These institutions are desig- 
nated by a check mark (“). 

Member colleges subscribing to the 
Candidates Reply Date agreement are 
indicated by a dot (*). These 120 in- 
stitutions have agreed not to require 
any candidate admitted as a freshman 
to give notice before May 18, 1955 of 
his decision to attend one of the sub- 
scribing colleges or to accept financial 
aid from it. Colleges listed with an as- 


“Lafayette College 
e Lake Forest College 
“Lehigh University 
e Lewis and Clark College 
“Manhattan College 
Manhattanville College of the 
Sacred Heart 
e Mary Baldwin College 
¢ Marymount College* 
¢ Marywood College* 
Massachusetts Institute of 
Technology 
© McGill University 
Michigan State College 
“Middlebury College 
e“Mills College 
“Mount Holyoke College 
“Muhlenberg College 
Newark College of Engineering 
e Newcomb College 
e“New Jersey College for Women 
New York University 
“Northwestern University 
“Oberlin College 
“Occidental College 
“Ohio Wesleyan University 
“Pembroke College in Brown 
University 
“Pennsylvania College for Women* 
Pennsylvania State University 
¢“Pomona College 
e“Princeton University 
¢ Providence College* 
“Radcliffe College 
“Randolph-Macon Woman’s College 
“Reed College 
“Regis College 
¢”Rensselaer Polytechnic Institute 
“Rollins College 
¢ Rosemont College* 
¢ Russell Sage College 
“Rutgers University 
¢ Saint Joseph College (Connecticut) 
¢ Saint Joseph College (Maryland) 
e St. Joseph’s College for Women 
“St. Lawrence University 
e Saint Mary’s College 


terisk (*) except scholarship appli- 
cants from the Candidates Reply Date. 
The Candidates Reply Date, origi- 
nally known as the Uniform Accept- 
ance Date, was first used in 1948 when 
eight colleges signed the agreement for 
that year. It was inaugurated in order 
to give candidates ample time to con- 
sider all opportunities available to 
them. Now, as in earlier years, how- 
ever, the candidate is encouraged to 
notify all institutions to which he has 
applied as soon as he definitely decides 
which college he wishes to attend. 


¢ Salem College 
°“Scripps College 
e Seton Hill College 
“Simmons College 
e“Skidmore College 
e“Smith College 
e”Stanford University 
e Stevens Institute of Technology 
“Swarthmore College 
“Sweet Briar College 
e Syracuse University 
“Trinity College (Connecticut) 
¢ Trinity College (Washington, D. C.) 
°*Tufts College 
“Union College 
University of California 
¢ University of Chicago 
© University of Colorado 
University of Connecticut 
University of Denver 
“University of Massachusetts 
e¢ University of Michigan 
University of Notre Dame 
e”“University of Pennsylvania 
“University of Redlands 
“University of Rochester 
¢”“University of Southern California 
© University of the South* 
University of Vermont and State 
Agricultural College 
© University of Virginia 
¢ Ursinus College 
“Vassar College 
e Villanova University* 
e“Wagner College 
¢ Washington and Jefferson College* 
¢ Washington and Lee University 
“Wellesley College 
“Wells College 
“Wesleyan University 
e Western Reserve University 
“Wheaton College 
e“Whitman College 
“Whittier College 
“Williams College 
“Wilson College 
“Yale University 





NEWS OF THE COLLEGE SCHOLARSHIP SERVICE 
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Early activities 


First reports: By mid-February, the 
parents of 10,000 scholarship appli- 
cants had sent their College Scholar- 
ship Service financial statements to 
the Service’s offices in Princeton and 
Los Angeles. There photographic cop- 
ies were made and sent to those col- 
leges named by the parents on the 
statement. 

A survey of the 8,000 statements re- 
ceived by the end of January showed 
that the average number of colleges 
listed to receive copies was 2.39 in the 
Princeton service area and 2.4] in 
Los Angeles. These figures correspond 
closely to the average number of col- 
leges listed by the entire College Board 
candidate group to receive test score 
reports. The scholarship-candidate 
ratio was expected to increase only 
slightly if at all as deadlines for finan- 
cial aid applications drew closer. 

Before copies of the statements are 
sent to colleges they are scanned for 
omissions and apparent inaccuracies 
and these are indicated by the Service 
on the form. The parents are requested 
to clarify such items and their explana- 
tions are transmitted to the colleges. 
The first scanning experience by a 
small staff working on a part-time 
basis showed that 10 statements an 
hour could be inspected. The record 
was 112 forms for one person during 
an eight-hour day. About one-quarter 
of the cases required follow-up corre- 
spondence with the parents in order to 
complete the information requested or 
eliminate ambiguities. The parents’ 
responses to these requests were made 
with good will and care. 


Information available: A prelimin- 
ary version of a Css manual which out- 
lines procedures which may be used in 
estimating the financial need of stu- 
dents has been distributed to colleges 
participating in the Service. Other ac- 
tivities have included meetings by sev- 
eral groups of college scholarship and 
admissions officers to discuss compu- 
tational methods and the scheduling 
of a general meeting by the Service 


on April 5 at which college representa- 
tives may report their experiences and 
present their comments and criticisms 
of the Service. 

The css welcomes requests from col- 
leges for information concerning the 
use of the financial statement. Inquir- 
ies should be addressed to the College 
Scholarship Service, Box 176, Prince- 
ton, New Jersey, or Box 27896, Los 
Angeles, California. A general descrip- 
tion of the Service’s purpose and activ- 
ities is also available in a leaflet which 
will be sent to school and college of- 
ficers pn request. 


List of applicants: The Service has 
been requested by several small groups 
of colleges to prepare lists of their 
joint scholarship applicants. Lists of 


this kind, which have been prepared in 
the past by some colleges with tradi- 
tional ties, assist the scholarship of- 
ficers to meet and discuss the financial 
needs of the applicants they have in 
common before notifying them of 
awards. The lists prepared by the css 
will include all applicants whose par- 
ents submitted financial statements by 
March 7. Other colleges are expected 
to prepare their own lists of joint can- 
didates and to hold meetings at which 
their applications will be considered. 

In September, after all awards to 
the class entering in 1955 have been 
made, the css will send a list to each 
participating college of all candidates 
who applied to it for financial aid and 
ask for a report of the action taken on 
each candidate’s application. These 
reports will be used to compile lists for 
each college of the kinds and amounts 
of aid offered to and accepted by those 
applicants which it had in common 
with other participating colleges. 


Increased participation: The possi- 
bility of opening the Service to colleges 
which are not members of the College 
Board and to responsible sponsored 
scholarship programs will be consid- 
ered this spring. Ten non-member col- 
leges and a number of organizations 
have expressed interest in the program. 
In any case, the number of participat- 
ing member colleges, now 95 (see page 
21), is expected to increase before the 
beginning of the new academic year 
in September. 


Upperclass form: A special adapta- 
tion of the css form has been made 
available to participating colleges at 
their request for use in annual reviews 
of upperclass scholarship awards. 
More than 7,000 copies of the form 
have been ordered by 30 colleges. 


Future possibilities 


Computation service: The css hopes 
to be able to assist colleges during the 
1955-56 academic year in their compu- 
tations of the amount of financial aid 
needed by applicants. Details of this 
service must await committee study 
and information which will be yielded 
by the first year’s operation. Possibili- 
ties range from such detailed and spe- 
cific work as actual estimates of the 
amounts of financial support to be rea- 
sonably expected from the families of 
individual applicants to more general 
summaries of key information ob- 
tained from each financial statement 
accompanied by tables of normative 
data relating to a large variety of fam- 
ily situations and financial factors. 


Computation manual: A new com- 
putation manual to improve on the pre- 
liminary edition distributed to partic- 
ipating colleges this year will be pre- 
pared during the summer. The manual 
will be planned to accommodate any 
changes which may be made in the 
computation procedure as a result of 
the colleges’ experience with it in 1954- 
55 and any improvements suggested 
by data now being collected from the 
financial statements themselves. 





